-- Logs begin at Wed 2022-07-13 21:25:44 UTC, end at Wed 2022-07-13 22:27:19 UTC. -- Jul 13 21:57:52.412718 dv-cinder-799871-28-iscsi systemd[1]: Starting Devstack devstack@keystone.service... Jul 13 21:57:52.421947 dv-cinder-799871-28-iscsi devstack@keystone.service[80368]: [uWSGI] getting INI configuration from /etc/keystone/keystone-uwsgi-public.ini Jul 13 21:57:52.425436 dv-cinder-799871-28-iscsi devstack@keystone.service[80368]: *** Starting uWSGI 2.0.18-debian (64bit) on [Wed Jul 13 21:57:52 2022] *** Jul 13 21:57:52.425436 dv-cinder-799871-28-iscsi devstack@keystone.service[80368]: compiled with version: 10.0.1 20200405 (experimental) [master revision 0be9efad938:fcb98e4978a:705510a708d3642c9c962beb663c476167e4e8a4] on 11 April 2020 11:15:55 Jul 13 21:57:52.425436 dv-cinder-799871-28-iscsi devstack@keystone.service[80368]: os: Linux-5.4.0-72-generic #80-Ubuntu SMP Mon Apr 12 17:35:00 UTC 2021 Jul 13 21:57:52.425436 dv-cinder-799871-28-iscsi devstack@keystone.service[80368]: nodename: dv-cinder-799871-28-iscsi Jul 13 21:57:52.425436 dv-cinder-799871-28-iscsi devstack@keystone.service[80368]: machine: x86_64 Jul 13 21:57:52.425436 dv-cinder-799871-28-iscsi devstack@keystone.service[80368]: clock source: unix Jul 13 21:57:52.425436 dv-cinder-799871-28-iscsi devstack@keystone.service[80368]: pcre jit disabled Jul 13 21:57:52.425436 dv-cinder-799871-28-iscsi devstack@keystone.service[80368]: detected number of CPU cores: 4 Jul 13 21:57:52.425436 dv-cinder-799871-28-iscsi devstack@keystone.service[80368]: current working directory: / Jul 13 21:57:52.425436 dv-cinder-799871-28-iscsi devstack@keystone.service[80368]: detected binary path: /usr/bin/uwsgi-core Jul 13 21:57:52.425436 dv-cinder-799871-28-iscsi devstack@keystone.service[80368]: your processes number limit is 47680 Jul 13 21:57:52.425436 dv-cinder-799871-28-iscsi devstack@keystone.service[80368]: your memory page size is 4096 bytes Jul 13 21:57:52.425436 dv-cinder-799871-28-iscsi devstack@keystone.service[80368]: detected max file descriptor number: 2048 Jul 13 21:57:52.425436 dv-cinder-799871-28-iscsi devstack@keystone.service[80368]: lock engine: pthread robust mutexes Jul 13 21:57:52.425436 dv-cinder-799871-28-iscsi devstack@keystone.service[80368]: thunder lock: enabled Jul 13 21:57:52.425436 dv-cinder-799871-28-iscsi devstack@keystone.service[80368]: uwsgi socket 0 bound to UNIX address /var/run/uwsgi/keystone-wsgi-public.socket fd 4 Jul 13 21:57:52.425436 dv-cinder-799871-28-iscsi devstack@keystone.service[80368]: Python version: 3.8.10 (default, Mar 15 2022, 12:22:08) [GCC 9.4.0] Jul 13 21:57:52.469361 dv-cinder-799871-28-iscsi devstack@keystone.service[80368]: Python main interpreter initialized at 0x5613119239c0 Jul 13 21:57:52.469361 dv-cinder-799871-28-iscsi devstack@keystone.service[80368]: python threads support enabled Jul 13 21:57:52.469361 dv-cinder-799871-28-iscsi devstack@keystone.service[80368]: your server socket listen backlog is limited to 100 connections Jul 13 21:57:52.469361 dv-cinder-799871-28-iscsi devstack@keystone.service[80368]: your mercy for graceful operations on workers is 90 seconds Jul 13 21:57:52.469746 dv-cinder-799871-28-iscsi devstack@keystone.service[80368]: mapped 403077 bytes (393 KB) for 2 cores Jul 13 21:57:52.469746 dv-cinder-799871-28-iscsi devstack@keystone.service[80368]: *** Operational MODE: preforking *** Jul 13 21:57:52.470060 dv-cinder-799871-28-iscsi devstack@keystone.service[80368]: *** uWSGI is running in multiple interpreter mode *** Jul 13 21:57:52.470060 dv-cinder-799871-28-iscsi devstack@keystone.service[80368]: spawned uWSGI master process (pid: 80368) Jul 13 21:57:52.470293 dv-cinder-799871-28-iscsi systemd[1]: Started Devstack devstack@keystone.service. Jul 13 21:57:52.470837 dv-cinder-799871-28-iscsi devstack@keystone.service[80368]: spawned uWSGI worker 1 (pid: 80381, cores: 1) Jul 13 21:57:52.471173 dv-cinder-799871-28-iscsi devstack@keystone.service[80368]: spawned uWSGI worker 2 (pid: 80382, cores: 1) Jul 13 21:57:52.472281 dv-cinder-799871-28-iscsi devstack@keystone.service[80368]: running "unix_signal:15 gracefully_kill_them_all" (master-start)... Jul 13 21:57:57.816572 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 13 21:57:57.816572 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: from cryptography.utils import int_from_bytes Jul 13 21:57:57.817374 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: /usr/lib/python3/dist-packages/secretstorage/dhcrypto.py:15: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 13 21:57:57.817374 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: from cryptography.utils import int_from_bytes Jul 13 21:57:57.817780 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 13 21:57:57.817780 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: from cryptography.utils import int_from_bytes Jul 13 21:57:57.817886 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: /usr/lib/python3/dist-packages/secretstorage/util.py:19: CryptographyDeprecationWarning: int_from_bytes is deprecated, use int.from_bytes instead Jul 13 21:57:57.817886 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: from cryptography.utils import int_from_bytes Jul 13 21:57:57.865014 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jul 13 21:57:57.865184 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jul 13 21:57:57.865339 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] command line args: [] {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jul 13 21:57:57.865595 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2589}} Jul 13 21:57:57.865728 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jul 13 21:57:57.865817 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] Configuration options gathered from: {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2590}} Jul 13 21:57:57.865886 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] ================================================================================ {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jul 13 21:57:57.865946 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] command line args: [] {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2591}} Jul 13 21:57:57.866004 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] admin_token = **** {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 21:57:57.866230 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] config files: ['/etc/keystone/keystone.conf'] {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2592}} Jul 13 21:57:57.866363 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] config_dir = [] {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 21:57:57.866452 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] ================================================================================ {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2594}} Jul 13 21:57:57.866544 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 21:57:57.866607 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] config_source = [] {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 21:57:57.866665 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] admin_token = **** {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 21:57:57.866797 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] debug = True {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 21:57:57.866945 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] config_dir = [] {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 21:57:57.867021 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 21:57:57.867214 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] config_file = ['/etc/keystone/keystone.conf'] {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 21:57:57.867214 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] config_source = [] {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 21:57:57.867325 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 21:57:57.867395 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] debug = True {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 21:57:57.867461 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 21:57:57.867628 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] default_log_levels = ['amqp=WARN', 'amqplib=WARN', 'boto=WARN', 'qpid=WARN', 'sqlalchemy=WARN', 'suds=INFO', 'oslo.messaging=INFO', 'oslo_messaging=INFO', 'iso8601=WARN', 'requests.packages.urllib3.connectionpool=WARN', 'urllib3.connectionpool=WARN', 'websocket=WARN', 'requests.packages.urllib3.util.retry=WARN', 'urllib3.util.retry=WARN', 'keystonemiddleware=WARN', 'routes.middleware=WARN', 'stevedore=WARN', 'taskflow=WARN', 'keystoneauth=WARN', 'oslo.cache=INFO', 'oslo_policy=INFO', 'dogpile.core.dogpile=INFO', 'dogpile=INFO', 'routes=INFO'] {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 21:57:57.867730 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 21:57:57.867836 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] default_publisher_id = None {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 21:57:57.867922 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 21:57:57.867984 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] insecure_debug = False {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 21:57:57.868297 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] list_limit = None {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 21:57:57.868418 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] instance_format = [instance: %(uuid)s] {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 21:57:57.868491 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] log_config_append = None {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 21:57:57.868597 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] instance_uuid_format = [instance: %(uuid)s] {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 21:57:57.868702 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 21:57:57.868767 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] list_limit = None {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 21:57:57.868825 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] log_dir = None {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 21:57:57.868882 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] log_config_append = None {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 21:57:57.868945 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] log_file = None {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 21:57:57.869111 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] log_date_format = %Y-%m-%d %H:%M:%S {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 21:57:57.869200 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 21:57:57.869263 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] log_dir = None {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 21:57:57.869327 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 21:57:57.869387 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] log_file = None {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 21:57:57.869582 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 21:57:57.869697 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] log_rotate_interval = 1 {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 21:57:57.869781 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 21:57:57.869848 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] log_rotate_interval_type = days {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 21:57:57.869962 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 21:57:57.870107 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] log_rotation_type = none {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 21:57:57.870186 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 21:57:57.870312 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] logging_context_format_string = %(color)s%(levelname)s %(name)s [%(global_request_id)s %(request_id)s %(project_name)s %(user_name)s%(color)s] %(instance)s%(color)s%(message)s {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 21:57:57.870463 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 21:57:57.870637 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] logging_debug_format_suffix = {{(pid=%(process)d) %(funcName)s %(pathname)s:%(lineno)d}} {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 21:57:57.870726 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 21:57:57.870801 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] logging_default_format_string = %(color)s%(levelname)s %(name)s [-%(color)s] %(instance)s%(color)s%(message)s {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 21:57:57.870863 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 21:57:57.870946 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] logging_exception_prefix = ERROR %(name)s %(instance)s {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 21:57:57.871018 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 21:57:57.871146 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] logging_user_identity_format = %(user)s %(project)s %(domain)s %(system_scope)s %(user_domain)s %(project_domain)s {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 21:57:57.871252 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 21:57:57.871468 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] max_logfile_count = 30 {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 21:57:57.871592 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 21:57:57.871674 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] max_logfile_size_mb = 200 {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 21:57:57.871738 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 21:57:57.871842 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] max_param_size = 64 {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 21:57:57.871994 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 21:57:57.872091 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] max_project_tree_depth = 5 {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 21:57:57.872156 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 21:57:57.872289 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] max_token_size = 16384 {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 21:57:57.872554 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] notification_format = cadf {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 21:57:57.872647 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.52/identity {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 21:57:57.872800 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] publish_errors = False {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 21:57:57.872928 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] notification_opt_out = ['identity.authenticate.success', 'identity.authenticate.pending', 'identity.authenticate.failed'] {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 21:57:57.873024 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 21:57:57.873248 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 21:57:57.873335 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] public_endpoint = http://10.222.0.52/identity {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 21:57:57.873408 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 21:57:57.873485 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] publish_errors = False {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 21:57:57.873651 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 21:57:57.873734 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] pydev_debug_host = None {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 21:57:57.873879 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 21:57:57.873968 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] pydev_debug_port = None {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 21:57:57.874103 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] rate_limit_burst = 0 {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 21:57:57.874187 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] standard_threads = False {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 21:57:57.874296 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] rate_limit_except_level = CRITICAL {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 21:57:57.874399 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 21:57:57.874568 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] rate_limit_interval = 0 {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 21:57:57.874648 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 21:57:57.874776 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 21:57:57.874852 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] standard_threads = False {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 21:57:57.874930 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] use_journal = False {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 21:57:57.875054 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] strict_password_check = False {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 21:57:57.875138 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] use_json = False {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 21:57:57.875212 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] use_stderr = False {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 21:57:57.875282 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] syslog_log_facility = LOG_USER {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 21:57:57.875353 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] use_syslog = False {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 21:57:57.875424 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] use_eventlog = False {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 21:57:57.875567 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] use_journal = False {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 21:57:57.875673 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 21:57:57.875750 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] use_json = False {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 21:57:57.875820 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.875897 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] use_stderr = False {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 21:57:57.876039 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.876132 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] use_syslog = False {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 21:57:57.876282 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] watch_log_file = False {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2602}} Jul 13 21:57:57.876370 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.876518 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.876599 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] oslo_concurrency.disable_process_locking = False {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.876763 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.876852 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] oslo_concurrency.lock_path = None {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.877000 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.877090 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] profiler.connection_string = messaging:// {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.877240 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.877323 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] profiler.enabled = False {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.877474 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.877557 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] profiler.es_doc_type = notification {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.877715 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] profiler.es_scroll_size = 10000 {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.877796 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.877959 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.878046 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] profiler.es_scroll_time = 2m {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.878191 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] profiler.filter_error_trace = False {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.878273 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.878409 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.878516 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] profiler.hmac_keys = SECRET_KEY {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.878679 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.878759 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] profiler.sentinel_service_name = mymaster {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.878913 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.878991 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] profiler.socket_timeout = 0.1 {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.879152 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] profiler.trace_sqlalchemy = False {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.879235 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.879392 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.879476 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_buffer_size = 1000 {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.879627 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.879710 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_enabled = False {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.879865 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_process_name = {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.879939 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.880110 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_socket_file = /var/tmp/metrics_collector.sock {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.880218 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.880342 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.880425 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] oslo_messaging_metrics.metrics_thread_stop_timeout = 10 {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.880576 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.880656 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] oslo_middleware.http_basic_auth_user_file = /etc/htpasswd {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.880823 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.880905 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] audit.namespace = openstack {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.881072 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] oslo_policy.enforce_new_defaults = False {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.881149 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.881315 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] oslo_policy.enforce_scope = False {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.881395 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.881555 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.881633 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] oslo_policy.policy_default_rule = default {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.881783 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.881859 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] oslo_policy.policy_dirs = ['policy.d'] {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.882009 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.882088 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] oslo_policy.policy_file = policy.yaml {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.882235 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.882312 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] oslo_policy.remote_content_type = application/x-www-form-urlencoded {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.882465 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.882616 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] oslo_policy.remote_ssl_ca_crt_file = None {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.882734 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.882818 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_crt_file = None {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.882976 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.883061 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] oslo_policy.remote_ssl_client_key_file = None {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.883203 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.883290 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] oslo_policy.remote_ssl_verify_server_crt = False {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.883428 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.883507 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] keystone_authtoken.admin_password = **** {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.883756 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] keystone_authtoken.admin_tenant_name = admin {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.883899 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.884068 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] keystone_authtoken.admin_token = **** {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.884296 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.884397 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] keystone_authtoken.admin_user = None {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.884476 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.884554 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] keystone_authtoken.auth_admin_prefix = {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.884627 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.884627 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.884766 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] keystone_authtoken.auth_host = 127.0.0.1 {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.884840 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.884917 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] keystone_authtoken.auth_port = 35357 {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.885072 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.885150 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] keystone_authtoken.auth_protocol = https {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.885296 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.885452 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] keystone_authtoken.auth_section = None {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.885527 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.885598 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] keystone_authtoken.auth_type = None {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.885734 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.885818 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] keystone_authtoken.auth_uri = None {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.885959 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.886049 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] keystone_authtoken.auth_version = None {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.886192 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.886270 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] keystone_authtoken.cache = None {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.886433 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.886561 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] keystone_authtoken.cafile = None {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.886689 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] keystone_authtoken.certfile = None {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.886780 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.886921 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.886998 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] keystone_authtoken.delay_auth_decision = False {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.887146 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.887256 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] keystone_authtoken.enforce_token_bind = permissive {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.887378 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.887490 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] keystone_authtoken.http_connect_timeout = None {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.887609 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.887711 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] keystone_authtoken.http_request_max_retries = 3 {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.887838 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] keystone_authtoken.identity_uri = None {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.887941 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.888063 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.888182 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] keystone_authtoken.include_service_catalog = True {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.888294 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.888372 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] keystone_authtoken.insecure = False {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.888527 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.888609 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] keystone_authtoken.interface = internal {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.888753 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.888843 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] keystone_authtoken.keyfile = None {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.888977 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.889056 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_conn_get_timeout = 10 {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.889207 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.889325 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_dead_retry = 300 {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.889434 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.889525 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_maxsize = 10 {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.889671 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_socket_timeout = 3 {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.889749 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.889904 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.889981 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] keystone_authtoken.memcache_pool_unused_timeout = 60 {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.890137 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] keystone_authtoken.memcache_secret_key = **** {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.890219 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.890362 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.890450 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] keystone_authtoken.memcache_security_strategy = None {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.890642 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.890723 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] keystone_authtoken.memcache_use_advanced_pool = True {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.890877 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.890957 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] keystone_authtoken.memcached_servers = None {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.891111 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] keystone_authtoken.region_name = None {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.891217 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.891375 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles = ['service'] {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.891455 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.891618 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] keystone_authtoken.service_token_roles_required = False {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.891716 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.891842 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] keystone_authtoken.service_type = None {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.891941 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.892091 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] keystone_authtoken.token_cache_time = 300 {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.892171 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.892332 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] keystone_authtoken.www_authenticate_uri = None {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.892410 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.892602 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] cache.backend = dogpile.cache.memcached {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.892682 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.892847 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] cache.backend_argument = **** {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.892924 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.893079 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.893155 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] cache.config_prefix = cache.oslo {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.893312 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.893391 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] cache.dead_timeout = 60.0 {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.893556 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] cache.debug_cache_backend = False {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.893663 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.893797 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] cache.enable_retry_client = False {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.893901 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.894035 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] cache.enable_socket_keepalive = False {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.894137 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.894378 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.894455 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] cache.enabled = True {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.894638 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.894718 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] cache.expiration_time = 600 {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.894874 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.894954 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] cache.hashclient_retry_attempts = 2 {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.895110 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.895188 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] cache.hashclient_retry_delay = 1.0 {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.895343 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.895421 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] cache.memcache_dead_retry = 300 {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.895576 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.895656 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] cache.memcache_pool_connection_get_timeout = 10 {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.895819 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] cache.memcache_pool_flush_on_reconnect = False {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.895904 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.896056 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.896134 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] cache.memcache_pool_maxsize = 10 {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.896295 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] cache.memcache_pool_unused_timeout = 60 {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.896380 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.896528 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.896605 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] cache.memcache_servers = ['localhost:11211'] {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.896769 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.896847 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] cache.memcache_socket_timeout = 1.0 {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.897001 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.897087 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] cache.proxies = [] {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.897224 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.897302 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] cache.retry_attempts = 2 {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.897456 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.897535 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] cache.retry_delay = 0.0 {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.897682 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.897770 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] cache.socket_keepalive_count = 1 {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.897909 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.897988 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] cache.socket_keepalive_idle = 1 {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.898135 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.898213 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] cache.socket_keepalive_interval = 1 {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.898368 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.898446 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] cache.tls_allowed_ciphers = None {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.898626 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] cache.tls_cafile = None {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.898711 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.898863 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] cache.tls_certfile = None {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.898941 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.899094 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.899180 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] cache.tls_enabled = False {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.899348 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] cache.tls_keyfile = None {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.899428 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.899593 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] application_credential.cache_time = None {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.899681 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.899831 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] application_credential.caching = True {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.899909 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.900106 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.900184 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] application_credential.driver = sql {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.900344 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.900422 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] application_credential.user_limit = -1 {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.900572 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] auth.external = None {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.900658 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] assignment.driver = sql {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.900799 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.900884 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] assignment.prohibited_implied_role = ['admin'] {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.901089 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.901172 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] auth.application_credential = None {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.901317 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.901403 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] auth.external = None {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.901544 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] auth.password = None {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.901635 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] auth.mapped = None {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.901770 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] auth.token = None {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.901855 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] auth.methods = ['external', 'password', 'token', 'oauth1', 'mapped', 'application_credential'] {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.902003 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.902082 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] auth.oauth1 = None {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.902237 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.902324 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] auth.password = None {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.902478 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.902590 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] auth.token = None {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.902727 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.902818 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] catalog.cache_time = None {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.902969 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.903057 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] catalog.caching = True {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.903215 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] catalog.driver = sql {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.903294 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.903440 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.903518 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] catalog.list_limit = None {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.903672 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] credential.caching = True {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.903760 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] catalog.template_file = default_catalog.templates {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.903912 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.903990 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] credential.auth_ttl = 15 {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.904141 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] credential.cache_time = None {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.904218 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.904382 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] credential.caching = True {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.904469 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.904612 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.904690 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] credential.driver = sql {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.904852 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.904931 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] credential.key_repository = /etc/keystone/credential-keys/ {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.905083 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.905169 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] credential.provider = fernet {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.905324 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] credential.user_limit = -1 {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.905404 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.905570 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] domain_config.cache_time = 300 {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.905654 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.905803 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.905894 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] domain_config.caching = True {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.906063 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.906143 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] domain_config.driver = sql {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.906313 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] endpoint_filter.driver = sql {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.906391 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.906580 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] endpoint_filter.return_all_endpoints_if_no_filter = True {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.906675 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.906839 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] endpoint_policy.driver = sql {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.906918 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.907075 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.907153 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] eventlet_server.admin_bind_host = 0.0.0.0 {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.907327 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.907415 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] eventlet_server.admin_port = 35357 {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.907560 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] federation.caching = True {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.907639 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] eventlet_server.public_bind_host = 0.0.0.0 {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.907796 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.907875 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] eventlet_server.public_port = 5000 {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.908035 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.908122 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] federation.assertion_prefix = {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.908277 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.908356 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] federation.caching = True {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.908503 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.908582 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] federation.default_authorization_ttl = 0 {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.908746 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.908833 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] federation.driver = sql {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.908977 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.909056 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] federation.federated_domain_name = Federated {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.909225 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] federation.remote_id_attribute = None {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.909303 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.909456 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.909542 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] federation.sso_callback_template = /etc/keystone/sso_callback_template.html {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.909700 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] federation.trusted_dashboard = [] {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.909778 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.909931 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.910009 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] fernet_receipts.key_repository = /etc/keystone/fernet-keys/ {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.910173 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.910253 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] fernet_receipts.max_active_keys = 3 {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.910406 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] identity.caching = True {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.910526 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] fernet_tokens.key_repository = /etc/keystone/fernet-keys/ {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.910672 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.910751 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] fernet_tokens.max_active_keys = 3 {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.910903 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.910986 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] identity.cache_time = 600 {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.911134 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.911214 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] identity.caching = True {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.911365 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.911445 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] identity.default_domain_id = default {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.911606 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.911686 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] identity.domain_config_dir = /etc/keystone/domains {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.911832 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.911911 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] identity.domain_configurations_from_database = False {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.912066 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.912145 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] identity.domain_specific_drivers_enabled = False {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.912312 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] identity.driver = sql {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.912391 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.912547 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] identity.list_limit = None {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.912626 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.912771 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.912849 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] identity.max_password_length = 4096 {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.912997 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.913077 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] identity.password_hash_algorithm = bcrypt {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.913222 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.913307 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] identity.password_hash_rounds = 4 {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.913464 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.913543 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] identity.salt_bytesize = None {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.913705 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] identity.scrypt_block_size = None {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.913783 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.913936 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] identity.scrypt_parallelism = None {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.914015 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.914184 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.914268 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] identity_mapping.backward_compatible_ids = True {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.914427 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.914535 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] identity_mapping.driver = sql {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.914709 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] identity_mapping.generator = sha256 {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.914791 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.915046 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.915136 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] jwt_tokens.jws_private_key_repository = /etc/keystone/jws-keys/private {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.915296 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.915477 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] jwt_tokens.jws_public_key_repository = /etc/keystone/jws-keys/public {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.915584 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.915747 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.915830 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] ldap.alias_dereferencing = default {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.915974 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.916062 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] ldap.auth_pool_connection_lifetime = 60 {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.916209 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.916290 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] ldap.auth_pool_size = 100 {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.916443 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] ldap.chase_referrals = None {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.916523 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.916681 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.916767 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] ldap.connection_timeout = -1 {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.916912 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] ldap.debug_level = None {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.916991 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.917140 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.917218 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] ldap.group_ad_nesting = False {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.917382 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] ldap.group_additional_attribute_mapping = [] {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.917468 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.917631 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.917751 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] ldap.group_attribute_ignore = [] {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.917862 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.918008 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] ldap.group_desc_attribute = description {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.918144 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] ldap.group_filter = None {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.918277 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.918410 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] ldap.group_id_attribute = cn {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.918615 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.918615 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.918856 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] ldap.group_member_attribute = member {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.918982 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.919114 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] ldap.group_members_are_ids = False {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.919238 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.919361 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] ldap.group_name_attribute = ou {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.919486 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.919608 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] ldap.group_objectclass = groupOfNames {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.919741 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.919873 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] ldap.group_tree_dn = None {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.919996 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.920125 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] ldap.page_size = 0 {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.920248 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.920373 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] ldap.password = **** {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.920497 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.920629 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] ldap.pool_connection_lifetime = 600 {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.920751 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.920873 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] ldap.pool_connection_timeout = -1 {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.920996 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.921120 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] ldap.pool_retry_delay = 0.1 {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.921241 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.921379 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] ldap.pool_retry_max = 3 {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.921513 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.921644 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] ldap.pool_size = 10 {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.921771 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.921893 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] ldap.query_scope = one {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.922016 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.922146 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] ldap.suffix = cn=example,cn=com {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.922269 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.922393 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] ldap.tls_cacertdir = None {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.922534 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.922664 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] ldap.tls_cacertfile = None {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.922796 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.922921 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] ldap.tls_req_cert = demand {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.923044 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] ldap.user = None {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.923166 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] ldap.url = ldap://localhost {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.923298 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.923423 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] ldap.use_auth_pool = True {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.923552 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.923686 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] ldap.use_pool = True {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.923808 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.923932 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] ldap.use_tls = False {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.924054 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.924177 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] ldap.user = None {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.924308 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.924435 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] ldap.user_additional_attribute_mapping = [] {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.924557 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.924682 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] ldap.user_attribute_ignore = ['default_project_id'] {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.924805 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.924937 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] ldap.user_default_project_id_attribute = None {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.925068 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.925194 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] ldap.user_description_attribute = description {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.925316 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.925439 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] ldap.user_enabled_attribute = enabled {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.925562 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.925686 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] ldap.user_enabled_default = True {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.925818 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.925942 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] ldap.user_enabled_emulation = False {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.926064 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.926191 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] ldap.user_enabled_emulation_dn = None {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.926315 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.926437 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] ldap.user_enabled_emulation_use_group_config = False {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.926593 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.926730 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] ldap.user_enabled_invert = False {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.926855 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.926977 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] ldap.user_enabled_mask = 0 {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.927100 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.927223 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] ldap.user_filter = None {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.927353 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.927477 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] ldap.user_id_attribute = cn {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.927598 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.927720 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] ldap.user_mail_attribute = mail {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.927844 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.927966 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] ldap.user_name_attribute = sn {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.928095 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.928095 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.928326 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] ldap.user_objectclass = inetOrgPerson {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.928458 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.928582 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] ldap.user_pass_attribute = userPassword {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.928703 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.928834 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] ldap.user_tree_dn = None {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.928959 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.929081 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] memcache.dead_retry = 300 {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.929202 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.929324 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] memcache.pool_connection_get_timeout = 10 {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.929446 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.929576 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] memcache.pool_maxsize = 10 {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.929699 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.929821 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] memcache.pool_unused_timeout = 60 {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.929944 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.930076 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] memcache.socket_timeout = 3 {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.930204 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.930334 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] oauth1.access_token_duration = 86400 {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.930457 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.930609 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] oauth1.driver = sql {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.930727 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.930853 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] oauth1.request_token_duration = 28800 {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.930975 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.931107 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] policy.driver = sql {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.931233 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.931357 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] policy.list_limit = None {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.931439 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.931600 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] receipt.cache_on_issue = True {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.931724 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.931869 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] receipt.cache_time = 300 {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.931995 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.932119 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] receipt.caching = True {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.932242 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] resource.caching = True {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.932366 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] receipt.expiration = 300 {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.932366 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] receipt.provider = fernet {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.932608 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.932730 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] resource.admin_project_domain_name = None {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.932730 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] resource.admin_project_name = None {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.932962 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.933086 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] resource.cache_time = None {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.933209 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.933341 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] resource.caching = True {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.933471 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.933603 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] resource.domain_name_url_safe = off {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.933727 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.933850 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] resource.driver = sql {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.933973 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.934104 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] resource.list_limit = None {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.934227 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.934350 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] resource.project_name_url_safe = off {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.934474 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.934612 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] revoke.cache_time = 3600 {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.934737 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.934938 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] revoke.caching = True {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.935064 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] role.caching = True {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.935189 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] revoke.driver = sql {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.935311 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] role.driver = sql {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.935443 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] revoke.expiration_buffer = 1800 {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.935566 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.935698 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] role.cache_time = None {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.935822 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.935944 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] role.caching = True {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.936066 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.936191 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] role.driver = sql {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.936315 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.936446 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] role.list_limit = None {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.936570 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.936694 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] saml.assertion_expiration_time = 3600 {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.936817 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.936941 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] saml.certfile = /etc/keystone/ssl/certs/signing_cert.pem {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.937082 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.937206 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] saml.idp_contact_company = Example, Inc. {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.937328 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.937453 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] saml.idp_contact_email = support@example.com {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.937578 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.937701 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] saml.idp_contact_name = SAML Identity Provider Support {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.937832 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.937956 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] saml.idp_contact_surname = Support {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.938078 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.938201 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] saml.idp_contact_telephone = +1 800 555 0100 {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.938325 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.938448 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] saml.idp_contact_type = other {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.938606 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.938742 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] saml.idp_entity_id = None {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.938865 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.938988 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] saml.idp_lang = en {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.939112 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.939233 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] saml.idp_metadata_path = /etc/keystone/saml2_idp_metadata.xml {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.939366 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.939490 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] saml.idp_organization_display_name = OpenStack SAML Identity Provider {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.939615 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.939739 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] saml.idp_organization_name = SAML Identity Provider {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.939862 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.939985 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] saml.idp_organization_url = https://example.com/ {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.940119 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.940244 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] saml.idp_sso_endpoint = None {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.940377 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.940502 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] saml.keyfile = /etc/keystone/ssl/private/signing_key.pem {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.940625 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.940746 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] saml.relay_state_prefix = ss:mem: {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.940878 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.941001 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] saml.xmlsec1_binary = xmlsec1 {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.941123 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.941123 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.941352 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] security_compliance.change_password_upon_first_use = False {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.941477 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.941609 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] security_compliance.disable_user_account_days_inactive = None {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.941731 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.941853 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] security_compliance.lockout_duration = 10 {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.941984 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.942106 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] security_compliance.lockout_failure_attempts = 2 {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.942239 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.942362 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] security_compliance.minimum_password_age = 0 {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.942527 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.942655 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] security_compliance.password_expires_days = None {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.942779 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.942902 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] security_compliance.password_regex = None {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.943033 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.943158 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] security_compliance.password_regex_description = None {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.943279 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.943401 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] security_compliance.unique_last_password_count = 2 {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.943524 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.943653 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] shadow_users.driver = sql {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.943793 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] token.caching = True {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.943916 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] token.allow_expired_window = 172800 {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.944038 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.944159 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] token.allow_rescope_scoped_token = True {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.944281 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.944403 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] token.cache_on_issue = True {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.944533 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.944658 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] token.cache_time = None {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.944779 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.944901 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] token.caching = True {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.945024 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.945147 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] token.expiration = 3600 {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.945278 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.945401 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] token.provider = fernet {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.945532 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.945677 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] token.revoke_by_id = True {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.945755 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.945829 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] tokenless_auth.issuer_attribute = SSL_CLIENT_I_DN {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.945911 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.945987 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] tokenless_auth.protocol = x509 {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.946060 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.946133 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] tokenless_auth.trusted_issuer = [] {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.946266 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.946342 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] totp.included_previous_windows = 1 {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.946413 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.946476 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] trust.allow_redelegation = False {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.946557 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.946618 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] trust.driver = sql {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.946678 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.946739 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] trust.max_redelegation_count = 3 {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.946857 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.946940 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] unified_limit.cache_time = None {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.947002 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.947074 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] unified_limit.caching = True {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.947146 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.947262 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] unified_limit.driver = sql {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.947385 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] database.connection = **** {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.947466 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] unified_limit.enforcement_model = flat {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.947548 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.947623 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] unified_limit.list_limit = None {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.947696 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.947767 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] wsgi.debug_middleware = False {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.947882 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.948009 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] database.backend = sqlalchemy {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.948089 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.948151 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] database.connection = **** {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.948225 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.948306 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] database.connection_debug = 0 {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.948369 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.948442 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] database.connection_parameters = {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.948514 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.948588 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] database.connection_recycle_time = 3600 {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.948662 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.948662 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.948830 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] database.connection_trace = False {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.948907 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.948969 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] database.db_inc_retry_interval = True {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.949092 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.949170 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] database.db_max_retries = 20 {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.949232 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.949643 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] database.db_max_retry_interval = 10 {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.949716 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.949832 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] database.db_retry_interval = 1 {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.949917 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.949992 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] database.max_overflow = 50 {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.950064 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.950141 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] database.max_pool_size = 5 {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.950213 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.950327 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] database.max_retries = 10 {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.950413 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.950577 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] database.mysql_enable_ndb = False {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.950657 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.950730 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] database.mysql_sql_mode = TRADITIONAL {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.950803 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80381) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jul 13 21:57:57.950877 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] database.pool_timeout = None {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.950877 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] database.retry_interval = 10 {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.950877 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] database.slave_connection = **** {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.951082 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] database.sqlite_synchronous = True {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.951269 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] database.use_db_reconnect = False {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2609}} Jul 13 21:57:57.951441 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG uwsgi [-] ******************************************************************************** {{(pid=80382) log_opt_values /usr/local/lib/python3.8/dist-packages/oslo_config/cfg.py:2613}} Jul 13 21:57:57.980582 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80381) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 13 21:57:57.980805 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.deleted`. {{(pid=80382) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 13 21:57:57.980894 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80381) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 13 21:57:57.981008 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80381) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 13 21:57:57.981091 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_user_delete_callback` subscribed to event `identity.user.disabled`. {{(pid=80382) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 13 21:57:57.981249 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [-] Callback: `keystone.application_credential.core.Manager._delete_app_creds_on_assignment_removal` subscribed to event `identity.remove_application_credentials_for_user.internal`. {{(pid=80382) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 13 21:57:57.989310 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80381) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 13 21:57:57.989651 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [-] Callback: `keystone.assignment.core.Manager._delete_domain_assignments` subscribed to event `identity.domain.deleted`. {{(pid=80382) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 13 21:57:58.017371 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80381) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 13 21:57:58.017576 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80381) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 13 21:57:58.017723 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.project.deleted`. {{(pid=80382) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 13 21:57:58.017832 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [-] Callback: `keystone.catalog.core.Manager._on_project_or_endpoint_delete` subscribed to event `identity.endpoint.deleted`. {{(pid=80382) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 13 21:57:58.162381 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80382) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 13 21:57:58.163140 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [-] Callback: `keystone.federation.core.Manager._cleanup_identity_provider` subscribed to event `identity.domain_deleted.internal`. {{(pid=80381) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 13 21:57:58.211160 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80382) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 13 21:57:58.211394 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80382) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 13 21:57:58.211543 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._domain_deleted` subscribed to event `identity.domain_deleted.internal`. {{(pid=80381) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 13 21:57:58.211812 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [-] Callback: `keystone.identity.core.Manager._unset_default_project` subscribed to event `identity.project.deleted`. {{(pid=80381) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 13 21:57:58.274935 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80381) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 13 21:57:58.275236 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80381) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 13 21:57:58.275370 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80381) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 13 21:57:58.275604 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80381) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 13 21:57:58.275728 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._trust_callback` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80382) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 13 21:57:58.275898 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80381) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 13 21:57:58.276028 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._consumer_callback` subscribed to event `identity.OS-OAUTH1:consumer.deleted`. {{(pid=80382) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 13 21:57:58.276151 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80381) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 13 21:57:58.276273 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.deleted`. {{(pid=80382) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 13 21:57:58.276273 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._project_callback` subscribed to event `identity.project.deleted`. {{(pid=80382) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 13 21:57:58.276555 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.user.disabled`. {{(pid=80382) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 13 21:57:58.276678 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [-] Callback: `keystone.revoke.core.Manager._user_callback` subscribed to event `identity.persist_revocation_event_for_user.internal`. {{(pid=80382) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 13 21:57:58.300973 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80381) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 13 21:57:58.301139 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80382) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 13 21:57:58.301366 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80381) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 13 21:57:58.301507 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.deleted`. {{(pid=80382) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 13 21:57:58.301593 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80381) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 13 21:57:58.301714 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.deleted`. {{(pid=80382) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 13 21:57:58.301796 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80381) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 13 21:57:58.301925 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.user.disabled`. {{(pid=80382) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 13 21:57:58.302010 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80381) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 13 21:57:58.302010 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80381) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 13 21:57:58.302200 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.domain.disabled`. {{(pid=80382) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 13 21:57:58.302320 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80381) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 13 21:57:58.302402 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.project.disabled`. {{(pid=80382) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 13 21:57:58.302402 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [-] Callback: `keystone.receipt.provider.Manager._drop_receipt_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80382) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 13 21:57:58.313410 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80381) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 13 21:57:58.313540 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [-] Callback: `keystone.trust.core.Manager._on_user_delete` subscribed to event `identity.user.deleted`. {{(pid=80382) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 13 21:57:58.326389 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80381) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 13 21:57:58.326636 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80381) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 13 21:57:58.326775 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.OS-TRUST:trust.deleted`. {{(pid=80382) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 13 21:57:58.326863 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80381) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 13 21:57:58.326928 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.deleted`. {{(pid=80382) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 13 21:57:58.327089 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80381) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 13 21:57:58.327215 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.deleted`. {{(pid=80382) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 13 21:57:58.327366 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80381) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 13 21:57:58.327449 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.user.disabled`. {{(pid=80382) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 13 21:57:58.327514 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80381) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 13 21:57:58.327634 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.domain.disabled`. {{(pid=80382) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 13 21:57:58.327708 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80381) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 13 21:57:58.327792 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.project.disabled`. {{(pid=80382) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 13 21:57:58.327956 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [-] Callback: `keystone.token.provider.Manager._drop_token_cache` subscribed to event `identity.invalidate_token_cache.internal`. {{(pid=80382) register_event_callback /opt/stack/keystone/keystone/notifications.py:295}} Jul 13 21:57:58.344782 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80381) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 21:57:58.344969 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80381) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 13 21:57:58.345084 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/projects' {}] {{(pid=80382) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 21:57:58.345251 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/projects' {}] {{(pid=80382) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 13 21:57:58.345449 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80381) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 21:57:58.345449 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80381) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 13 21:57:58.345671 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/domains' {}] {{(pid=80382) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 21:57:58.345765 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.common [-] Adding additional resource route (alternate) to API auth: ['/OS-FEDERATION/domains' {}] {{(pid=80382) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:433}} Jul 13 21:57:58.345979 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80381) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 21:57:58.346238 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80381) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 21:57:58.346361 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/system' {}] {{(pid=80382) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 21:57:58.346540 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80381) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 21:57:58.346674 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/catalog' {}] {{(pid=80382) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 21:57:58.346798 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80381) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 21:57:58.346957 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens/OS-PKI/revoked' {}] {{(pid=80382) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 21:57:58.347096 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth: ['/auth/tokens' {}] {{(pid=80382) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 21:57:58.357523 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80381) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 21:57:58.357796 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80381) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 21:57:58.357923 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2' {}] {{(pid=80382) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 21:57:58.358073 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80381) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 21:57:58.358218 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/saml2/ecp' {}] {{(pid=80382) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 21:57:58.358354 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80381) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 21:57:58.358480 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/websso/' {}] {{(pid=80382) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 21:57:58.358760 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.common [-] Adding resource routes to API auth/OS-FEDERATION: ['/auth/OS-FEDERATION/identity_providers//protocols//websso' {}] {{(pid=80382) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 21:57:58.366461 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80381) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 13 21:57:58.367011 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.common [-] Adding standard routes to API credentials for `MethodViewType` (API Prefix: ) [/credentials, /credentials/] {{(pid=80382) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 13 21:57:58.370093 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80381) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 13 21:57:58.370370 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80381) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 21:57:58.370582 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.common [-] Adding standard routes to API domains for `MethodViewType` (API Prefix: ) [/domains, /domains/] {{(pid=80382) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 13 21:57:58.370680 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80381) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 21:57:58.370820 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config' {}] {{(pid=80382) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 21:57:58.370962 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80381) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 21:57:58.371086 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config/' {}] {{(pid=80382) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 21:57:58.371207 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80381) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 21:57:58.371334 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//config//' {}] {{(pid=80382) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 21:57:58.371416 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80381) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 21:57:58.371625 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config/default' {}] {{(pid=80382) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 21:57:58.371750 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80381) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 21:57:58.371896 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config//default' {}] {{(pid=80382) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 21:57:58.372077 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80381) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 21:57:58.372200 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains/config///default' {}] {{(pid=80382) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 21:57:58.372323 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80381) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 21:57:58.372460 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles' {}] {{(pid=80382) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 21:57:58.372589 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80381) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 21:57:58.372716 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80381) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 21:57:58.372842 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//users//roles/' {}] {{(pid=80382) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 21:57:58.372968 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles' {}] {{(pid=80382) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 21:57:58.373144 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.common [-] Adding resource routes to API domains: ['/domains//groups//roles/' {}] {{(pid=80382) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 21:57:58.399078 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80381) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 21:57:58.399946 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.common [-] Adding resource routes to API ec2tokens: ['/ec2tokens' {}] {{(pid=80382) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 21:57:58.400841 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80381) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 13 21:57:58.401115 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80381) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 21:57:58.401568 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.common [-] Adding standard routes to API endpoints for `MethodViewType` (API Prefix: ) [/endpoints, /endpoints/] {{(pid=80382) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 13 21:57:58.401847 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.common [-] Adding resource routes to API endpoints: ['/endpoints//OS-ENDPOINT-POLICY/policy' {}] {{(pid=80382) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 21:57:58.406362 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80381) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 13 21:57:58.406659 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80381) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 21:57:58.406933 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80381) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 21:57:58.407187 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.common [-] Adding standard routes to API groups for `MethodViewType` (API Prefix: ) [/groups, /groups/] {{(pid=80382) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 13 21:57:58.407460 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users' {}] {{(pid=80382) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 21:57:58.407740 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.common [-] Adding resource routes to API groups: ['/groups//users/' {}] {{(pid=80382) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 21:57:58.414336 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80381) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 13 21:57:58.414581 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80381) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 21:57:58.415313 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.common [-] Adding standard routes to API limits for `MethodViewType` (API Prefix: ) [/limits, /limits/] {{(pid=80382) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 13 21:57:58.415694 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.common [-] Adding resource routes to API limits: ['/limits/model' {}] {{(pid=80382) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 21:57:58.419760 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80381) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 13 21:57:58.420042 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80381) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 21:57:58.420247 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-EP-FILTER for `MethodViewType` (API Prefix: /OS-EP-FILTER) [/endpoint_groups, /endpoint_groups/] {{(pid=80382) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 13 21:57:58.420375 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80381) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 21:57:58.420532 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoints//projects' {}] {{(pid=80382) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 21:57:58.420618 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80381) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 21:57:58.420779 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints/' {}] {{(pid=80382) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 21:57:58.420865 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80381) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 21:57:58.421055 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80381) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 21:57:58.421148 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoints' {}] {{(pid=80382) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 21:57:58.421306 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80381) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 21:57:58.421394 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/projects//endpoint_groups' {}] {{(pid=80382) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 21:57:58.421576 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80381) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 21:57:58.421661 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//endpoints' {}] {{(pid=80382) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 21:57:58.421838 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects' {}] {{(pid=80382) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 21:57:58.422103 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-EP-FILTER: ['/endpoint_groups//projects/' {}] {{(pid=80382) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 21:57:58.440381 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80381) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 21:57:58.440630 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80381) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 21:57:58.440946 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/saml2/metadata' {}] {{(pid=80382) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 21:57:58.441218 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-FEDERATION: ['/identity_providers//protocols//auth' {}] {{(pid=80382) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 21:57:58.445274 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80381) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 13 21:57:58.445807 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.common [-] Adding standard routes to API identity_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/identity_providers, /identity_providers/] {{(pid=80382) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 13 21:57:58.448920 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80381) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 21:57:58.449203 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80381) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 21:57:58.449877 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols/' {}] {{(pid=80382) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 21:57:58.450169 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.common [-] Adding resource routes to API protocols: ['/OS-FEDERATION/identity_providers//protocols' {}] {{(pid=80382) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 21:57:58.454688 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80381) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 13 21:57:58.455386 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.common [-] Adding standard routes to API mappings for `MethodViewType` (API Prefix: /OS-FEDERATION) [/mappings, /mappings/] {{(pid=80382) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 13 21:57:58.458296 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80381) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 13 21:57:58.459029 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.common [-] Adding standard routes to API service_providers for `MethodViewType` (API Prefix: /OS-FEDERATION) [/service_providers, /service_providers/] {{(pid=80382) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 13 21:57:58.462000 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80381) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 21:57:58.462279 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80381) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 21:57:58.462583 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80381) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 21:57:58.462816 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles//inherited_to_projects' {}] {{(pid=80382) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 21:57:58.462943 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80381) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 21:57:58.463162 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//groups//roles/inherited_to_projects' {}] {{(pid=80382) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 21:57:58.463290 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80381) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 21:57:58.463430 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80381) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 21:57:58.463569 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles//inherited_to_projects' {}] {{(pid=80382) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 21:57:58.463696 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/domains//users//roles/inherited_to_projects' {}] {{(pid=80382) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 21:57:58.463969 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//users//roles//inherited_to_projects' {}] {{(pid=80382) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 21:57:58.464240 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-INHERIT: ['/projects//groups//roles//inherited_to_projects' {}] {{(pid=80382) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 21:57:58.482564 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80381) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 13 21:57:58.482857 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80381) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 21:57:58.483088 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80381) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 21:57:58.483368 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80381) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 21:57:58.483645 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.common [-] Adding standard routes to API OS-OAUTH1 for `MethodViewType` (API Prefix: /OS-OAUTH1) [/consumers, /consumers/] {{(pid=80382) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 13 21:57:58.483931 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/request_token' {}] {{(pid=80382) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 21:57:58.484197 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/access_token' {}] {{(pid=80382) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 21:57:58.484462 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-OAUTH1: ['/authorize/' {}] {{(pid=80382) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 21:57:58.491268 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80381) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 21:57:58.492447 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.common [-] Adding resource routes to API events: ['/events' {}] {{(pid=80382) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 21:57:58.492963 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80381) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 21:57:58.493251 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80381) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 21:57:58.494187 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/ca' {}] {{(pid=80382) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 21:57:58.494460 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.common [-] Adding resource routes to API OS-SIMPLE-CERT: ['/OS-SIMPLE-CERT/certificates' {}] {{(pid=80382) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 21:57:58.496456 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80381) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 13 21:57:58.496730 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80381) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 21:57:58.497005 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80381) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 21:57:58.497353 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80381) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 21:57:58.497627 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80381) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 21:57:58.497752 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.common [-] Adding standard routes to API policy for `MethodViewType` (API Prefix: ) [/policies, /policies/] {{(pid=80382) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 13 21:57:58.497959 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints' {}] {{(pid=80382) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 21:57:58.498239 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/endpoints/' {}] {{(pid=80382) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 21:57:58.498519 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services/' {}] {{(pid=80382) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 21:57:58.498866 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.common [-] Adding resource routes to API policy: ['/policies//OS-ENDPOINT-POLICY/services//regions/' {}] {{(pid=80382) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 21:57:58.510989 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80381) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 13 21:57:58.511262 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80381) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 21:57:58.511538 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80381) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 21:57:58.511823 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80381) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 21:57:58.512097 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80381) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 21:57:58.512242 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.common [-] Adding standard routes to API projects for `MethodViewType` (API Prefix: ) [/projects, /projects/] {{(pid=80382) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 13 21:57:58.512383 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80381) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 21:57:58.512465 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags' {}] {{(pid=80382) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 21:57:58.512638 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80381) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 21:57:58.512764 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//tags/' {}] {{(pid=80382) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 21:57:58.513027 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles/' {}] {{(pid=80382) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 21:57:58.513304 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//users//roles' {}] {{(pid=80382) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 21:57:58.513594 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles/' {}] {{(pid=80382) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 21:57:58.513867 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.common [-] Adding resource routes to API projects: ['/projects//groups//roles' {}] {{(pid=80382) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 21:57:58.531687 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80381) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 13 21:57:58.533239 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.common [-] Adding standard routes to API regions for `MethodViewType` (API Prefix: ) [/regions, /regions/] {{(pid=80382) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 13 21:57:58.534956 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80381) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 13 21:57:58.536553 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.common [-] Adding standard routes to API registered_limit for `MethodViewType` (API Prefix: ) [/registered_limits, /registered_limits/] {{(pid=80382) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 13 21:57:58.538257 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80381) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 21:57:58.539920 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80381) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 21:57:58.541794 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_assignments: ['/role_assignments' {}] {{(pid=80382) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 21:57:58.541947 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80381) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 13 21:57:58.542234 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80381) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 21:57:58.542502 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80381) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 21:57:58.543471 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.common [-] Adding resource routes to API role_inferences: ['/role_inferences' {}] {{(pid=80382) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 21:57:58.545282 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.common [-] Adding standard routes to API roles for `MethodViewType` (API Prefix: ) [/roles, /roles/] {{(pid=80382) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 13 21:57:58.545542 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies' {}] {{(pid=80382) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 21:57:58.545853 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.common [-] Adding resource routes to API roles: ['/roles//implies/' {}] {{(pid=80382) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 21:57:58.549889 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80381) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 21:57:58.551518 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80381) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 13 21:57:58.553271 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.common [-] Adding resource routes to API s3tokens: ['/s3tokens' {}] {{(pid=80382) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 21:57:58.554744 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80381) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 21:57:58.554874 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.common [-] Adding standard routes to API services for `MethodViewType` (API Prefix: ) [/services, /services/] {{(pid=80382) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 13 21:57:58.555017 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80381) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 21:57:58.555264 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80381) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 21:57:58.555578 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80381) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 21:57:58.558186 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles' {}] {{(pid=80382) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 21:57:58.558470 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/users//roles/' {}] {{(pid=80382) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 21:57:58.558776 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles' {}] {{(pid=80382) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 21:57:58.559050 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.common [-] Adding resource routes to API system: ['/system/groups//roles/' {}] {{(pid=80382) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 21:57:58.565131 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80381) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 13 21:57:58.565408 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80381) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 21:57:58.565679 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80381) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 21:57:58.568348 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.common [-] Adding standard routes to API trusts for `MethodViewType` (API Prefix: /OS-TRUST) [/trusts, /trusts/] {{(pid=80382) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 13 21:57:58.568631 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles' {}] {{(pid=80382) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 21:57:58.568905 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.common [-] Adding resource routes to API trusts: ['/trusts//roles/' {}] {{(pid=80382) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 21:57:58.573584 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80381) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 13 21:57:58.573854 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80381) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 21:57:58.574129 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80381) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 21:57:58.574764 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80381) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 21:57:58.575025 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80381) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 21:57:58.575301 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80381) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 21:57:58.575569 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80381) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 21:57:58.575836 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80381) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 21:57:58.576204 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80381) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 21:57:58.576468 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80381) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 21:57:58.576732 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80381) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 21:57:58.576966 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80381) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 21:57:58.577229 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80381) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 21:57:58.577317 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.common [-] Adding standard routes to API users for `MethodViewType` (API Prefix: ) [/users, /users/] {{(pid=80382) _add_resources /opt/stack/keystone/keystone/server/flask/common.py:343}} Jul 13 21:57:58.577487 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80381) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 21:57:58.577582 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//password' {}] {{(pid=80382) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 21:57:58.577769 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//groups' {}] {{(pid=80382) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 21:57:58.578044 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//projects' {}] {{(pid=80382) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 21:57:58.578311 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2' {}] {{(pid=80382) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 21:57:58.578624 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//credentials/OS-EC2/' {}] {{(pid=80382) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 21:57:58.578883 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens' {}] {{(pid=80382) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 21:57:58.579156 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens/' {}] {{(pid=80382) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 21:57:58.579530 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles' {}] {{(pid=80382) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 21:57:58.579803 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//OS-OAUTH1/access_tokens//roles/' {}] {{(pid=80382) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 21:57:58.580082 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials' {}] {{(pid=80382) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 21:57:58.580347 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//application_credentials/' {}] {{(pid=80382) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 21:57:58.580623 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules' {}] {{(pid=80382) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 21:57:58.580889 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.common [-] Adding resource routes to API users: ['/users//access_rules/' {}] {{(pid=80382) _add_mapped_resources /opt/stack/keystone/keystone/server/flask/common.py:417}} Jul 13 21:57:58.615514 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x5613119239c0 pid: 80381 (default app) Jul 13 21:57:58.619114 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: WSGI app 0 (mountpoint='') ready in 6 seconds on interpreter 0x5613119239c0 pid: 80382 (default app) Jul 13 21:57:58.619653 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afdfccd-240a-47ea-91ff-e80a229d9da4 None None] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:57:58.619896 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afdfccd-240a-47ea-91ff-e80a229d9da4 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:57:58.620172 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3afdfccd-240a-47ea-91ff-e80a229d9da4 None None] PATH_INFO: `/v3` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:57:58.624677 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1/1] 10.222.0.52 () {50 vars in 791 bytes} [Wed Jul 13 21:57:58 2022] GET /identity/v3/ => generated 255 bytes in 9 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 21:58:06.302404 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3c3541-5f43-46fd-b8e1-451ed67ee6b0 None None] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:58:06.303356 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3c3541-5f43-46fd-b8e1-451ed67ee6b0 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:58:06.303356 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa3c3541-5f43-46fd-b8e1-451ed67ee6b0 None None] PATH_INFO: `/` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:58:06.310352 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1/2] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jul 13 21:58:06 2022] GET /identity => generated 270 bytes in 14 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 21:58:06.315854 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47db7d9b-2bc2-4895-8666-ceac213db86b None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:58:06.316317 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47db7d9b-2bc2-4895-8666-ceac213db86b None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:58:06.316410 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47db7d9b-2bc2-4895-8666-ceac213db86b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:58:06.384806 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: INFO dbcounter [None req-47db7d9b-2bc2-4895-8666-ceac213db86b None None] Registered counter for database keystone Jul 13 21:58:06.409722 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG oslo_db.sqlalchemy.engines [None req-47db7d9b-2bc2-4895-8666-ceac213db86b None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80381) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 13 21:58:06.422159 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writer thread running {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 13 21:58:06.599938 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG passlib.handlers.bcrypt [None req-47db7d9b-2bc2-4895-8666-ceac213db86b None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80381) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jul 13 21:58:06.600283 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG passlib.handlers.bcrypt [None req-47db7d9b-2bc2-4895-8666-ceac213db86b None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80381) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jul 13 21:58:06.656143 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-47db7d9b-2bc2-4895-8666-ceac213db86b None None] MFA Rules not processed for user `cc99c93da5074cf5a48bc2aa9a92c5e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:58:06.658717 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-47db7d9b-2bc2-4895-8666-ceac213db86b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:58:06.802168 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 2/3] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jul 13 21:58:06 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 487 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jul 13 21:58:07.041373 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3c3541-5f43-46fd-b8e1-451ed67ee6b0 None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:58:07.121523 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: INFO dbcounter [None req-aa3c3541-5f43-46fd-b8e1-451ed67ee6b0 None None] Registered counter for database keystone Jul 13 21:58:07.143544 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG oslo_db.sqlalchemy.engines [None req-aa3c3541-5f43-46fd-b8e1-451ed67ee6b0 None None] MySQL server mode set to STRICT_TRANS_TABLES,STRICT_ALL_TABLES,NO_ZERO_IN_DATE,NO_ZERO_DATE,ERROR_FOR_DIVISION_BY_ZERO,TRADITIONAL,NO_ENGINE_SUBSTITUTION {{(pid=80382) _check_effective_sql_mode /usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/engines.py:314}} Jul 13 21:58:07.153758 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writer thread running {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:99}} Jul 13 21:58:07.273251 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa3c3541-5f43-46fd-b8e1-451ed67ee6b0 None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:58:07.278890 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4d9676b-1e37-4358-80e9-54575e72b48c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:58:07.279547 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d9676b-1e37-4358-80e9-54575e72b48c admin admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:58:07.279821 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d9676b-1e37-4358-80e9-54575e72b48c admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:58:07.280073 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4d9676b-1e37-4358-80e9-54575e72b48c admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:58:07.289035 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d9676b-1e37-4358-80e9-54575e72b48c admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:58:07.340709 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: WARNING keystone.common.rbac_enforcer.enforcer [None req-c4d9676b-1e37-4358-80e9-54575e72b48c admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jul 13 21:58:07.341280 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4d9676b-1e37-4358-80e9-54575e72b48c admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:58:07.351256 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 2/4] 10.222.0.52 () {58 vars in 1324 bytes} [Wed Jul 13 21:58:07 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 13 21:58:07.357826 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47db7d9b-2bc2-4895-8666-ceac213db86b None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:58:07.373305 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47db7d9b-2bc2-4895-8666-ceac213db86b None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:58:07.378186 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-394c2a24-4cc4-405d-ad62-c39beada00f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:58:07.379353 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394c2a24-4cc4-405d-ad62-c39beada00f2 admin admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:58:07.379631 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394c2a24-4cc4-405d-ad62-c39beada00f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:58:07.379902 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-394c2a24-4cc4-405d-ad62-c39beada00f2 admin admin] PATH_INFO: `/v3/projects/0b902d068f594f05b4de544301553ded` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:58:07.381429 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394c2a24-4cc4-405d-ad62-c39beada00f2 admin admin] RBAC: Authorizing `identity:get_project(project_id=0b902d068f594f05b4de544301553ded)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:58:07.434377 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: WARNING keystone.common.rbac_enforcer.enforcer [None req-394c2a24-4cc4-405d-ad62-c39beada00f2 admin admin] Deprecated policy rules found. Use oslopolicy-policy-generator and oslopolicy-policy-upgrade to detect and resolve deprecated policies in your configuration. Jul 13 21:58:07.434739 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-394c2a24-4cc4-405d-ad62-c39beada00f2 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:58:07.439416 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 3/5] 10.222.0.52 () {56 vars in 1207 bytes} [Wed Jul 13 21:58:07 2022] GET /identity/v3/projects/0b902d068f594f05b4de544301553ded => generated 344 bytes in 82 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 21:58:09.427641 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e738d1-a35c-4bc0-9c3e-76143a747bbc None None] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:58:09.428530 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e738d1-a35c-4bc0-9c3e-76143a747bbc None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:58:09.428530 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73e738d1-a35c-4bc0-9c3e-76143a747bbc None None] PATH_INFO: `/` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:58:09.434288 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 3/6] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jul 13 21:58:09 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 21:58:09.439904 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59035f55-a207-4e92-9058-408561ed18b8 None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:58:09.440317 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59035f55-a207-4e92-9058-408561ed18b8 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:58:09.440401 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59035f55-a207-4e92-9058-408561ed18b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:58:09.460562 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-59035f55-a207-4e92-9058-408561ed18b8 None None] MFA Rules not processed for user `cc99c93da5074cf5a48bc2aa9a92c5e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:58:09.462077 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-59035f55-a207-4e92-9058-408561ed18b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:58:09.470762 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 4/7] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jul 13 21:58:09 2022] POST /identity/v3/auth/tokens => generated 914 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 384 bytes (1 switches on core 0) Jul 13 21:58:09.492945 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e738d1-a35c-4bc0-9c3e-76143a747bbc None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:58:09.515840 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73e738d1-a35c-4bc0-9c3e-76143a747bbc None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:58:09.525132 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6d22779-cba5-4a9c-9af4-6c6f1f30d1b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:58:09.526354 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d22779-cba5-4a9c-9af4-6c6f1f30d1b9 admin admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:58:09.526703 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d22779-cba5-4a9c-9af4-6c6f1f30d1b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:58:09.526987 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6d22779-cba5-4a9c-9af4-6c6f1f30d1b9 admin admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:58:09.536199 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d22779-cba5-4a9c-9af4-6c6f1f30d1b9 admin admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:58:09.538011 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6d22779-cba5-4a9c-9af4-6c6f1f30d1b9 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:58:09.548710 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 4/8] 10.222.0.52 () {58 vars in 1324 bytes} [Wed Jul 13 21:58:09 2022] GET /identity/v3/auth/tokens => generated 914 bytes in 56 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 13 21:58:09.556359 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59035f55-a207-4e92-9058-408561ed18b8 None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:58:09.566100 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59035f55-a207-4e92-9058-408561ed18b8 None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:58:09.571821 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbf9cb97-1a7c-411b-955d-b1b45d0a1777 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:58:09.572792 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf9cb97-1a7c-411b-955d-b1b45d0a1777 admin admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:58:09.573069 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf9cb97-1a7c-411b-955d-b1b45d0a1777 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:58:09.573341 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbf9cb97-1a7c-411b-955d-b1b45d0a1777 admin admin] PATH_INFO: `/v3/users/cc99c93da5074cf5a48bc2aa9a92c5e7` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:58:09.575266 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf9cb97-1a7c-411b-955d-b1b45d0a1777 admin admin] RBAC: Authorizing `identity:get_user(user_id=cc99c93da5074cf5a48bc2aa9a92c5e7)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:58:09.577119 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbf9cb97-1a7c-411b-955d-b1b45d0a1777 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:58:09.578746 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 5/9] 10.222.0.52 () {56 vars in 1198 bytes} [Wed Jul 13 21:58:09 2022] GET /identity/v3/users/cc99c93da5074cf5a48bc2aa9a92c5e7 => generated 247 bytes in 23 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 21:58:11.625851 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960bd53b-d81c-434b-999b-32270d02ac75 None None] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:58:11.626417 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960bd53b-d81c-434b-999b-32270d02ac75 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:58:11.626417 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960bd53b-d81c-434b-999b-32270d02ac75 None None] PATH_INFO: `/` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:58:11.627791 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 5/10] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jul 13 21:58:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 21:58:11.633038 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316f5826-0c1d-43ef-8d06-21309301d476 None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:58:11.633491 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316f5826-0c1d-43ef-8d06-21309301d476 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:58:11.633585 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-316f5826-0c1d-43ef-8d06-21309301d476 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:58:11.651250 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-316f5826-0c1d-43ef-8d06-21309301d476 None None] MFA Rules not processed for user `cc99c93da5074cf5a48bc2aa9a92c5e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:58:11.652542 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-316f5826-0c1d-43ef-8d06-21309301d476 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:58:11.669084 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f2aaff-d9d1-41e0-add4-f573d59ceafb None None] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:58:11.669365 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f2aaff-d9d1-41e0-add4-f573d59ceafb None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:58:11.669640 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94f2aaff-d9d1-41e0-add4-f573d59ceafb None None] PATH_INFO: `/` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:58:11.671032 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 6/11] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jul 13 21:58:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 13 21:58:11.675645 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f4a088-7cce-49fb-98b1-6ef32d04a19e None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:58:11.675925 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f4a088-7cce-49fb-98b1-6ef32d04a19e None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:58:11.676200 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46f4a088-7cce-49fb-98b1-6ef32d04a19e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:58:11.719442 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG passlib.handlers.bcrypt [None req-46f4a088-7cce-49fb-98b1-6ef32d04a19e None None] detected 'bcrypt' backend, version '3.2.2' {{(pid=80382) _load_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:625}} Jul 13 21:58:11.719855 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG passlib.handlers.bcrypt [None req-46f4a088-7cce-49fb-98b1-6ef32d04a19e None None] 'bcrypt' backend lacks $2$ support, enabling workaround {{(pid=80382) _finalize_backend_mixin /usr/local/lib/python3.8/dist-packages/passlib/handlers/bcrypt.py:406}} Jul 13 21:58:11.738903 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 6/12] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jul 13 21:58:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 13 21:58:11.764998 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316f5826-0c1d-43ef-8d06-21309301d476 None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:58:11.765708 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-46f4a088-7cce-49fb-98b1-6ef32d04a19e None None] MFA Rules not processed for user `cc99c93da5074cf5a48bc2aa9a92c5e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:58:11.767421 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-46f4a088-7cce-49fb-98b1-6ef32d04a19e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:58:11.789768 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:58:11.869952 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 7/13] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jul 13 21:58:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 13 21:58:11.872245 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-316f5826-0c1d-43ef-8d06-21309301d476 None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:58:11.891802 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f4a088-7cce-49fb-98b1-6ef32d04a19e None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:58:11.906245 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7140ba9f-f03b-4d6a-8dc1-a246d260b20e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:58:11.907083 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7140ba9f-f03b-4d6a-8dc1-a246d260b20e None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:58:11.907360 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7140ba9f-f03b-4d6a-8dc1-a246d260b20e None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:58:11.907633 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7140ba9f-f03b-4d6a-8dc1-a246d260b20e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:58:11.918732 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7140ba9f-f03b-4d6a-8dc1-a246d260b20e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 21:58:11.929852 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application [None req-7140ba9f-f03b-4d6a-8dc1-a246d260b20e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 13 21:58:11.929852 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 21:58:11.929852 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 21:58:11.929852 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 21:58:11.929852 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 21:58:11.929852 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 21:58:11.929852 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 21:58:11.929852 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 21:58:11.929852 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 21:58:11.929852 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 21:58:11.929852 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 21:58:11.929852 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 21:58:11.929852 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 21:58:11.929852 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 21:58:11.929852 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 21:58:11.929852 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 21:58:11.930813 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 21:58:11.930813 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 21:58:11.930813 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 21:58:11.930813 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 21:58:11.930813 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 21:58:11.930813 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 21:58:11.930813 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 21:58:11.930813 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 21:58:11.930813 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 21:58:11.930813 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 21:58:11.930813 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 21:58:11.930813 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application with Lock( Jul 13 21:58:11.930813 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 21:58:11.930813 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._enter() Jul 13 21:58:11.930813 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 21:58:11.930813 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 21:58:11.931652 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 21:58:11.931652 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.creator() Jul 13 21:58:11.931652 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 21:58:11.931652 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application created_value = creator( Jul 13 21:58:11.931652 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 21:58:11.931652 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 21:58:11.931652 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 21:58:11.931652 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 21:58:11.931652 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 21:58:11.931652 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 21:58:11.931652 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 13 21:58:11.931652 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Jul 13 21:58:11.931652 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 7/14] 10.222.0.52 () {56 vars in 1096 bytes} [Wed Jul 13 21:58:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 13 21:58:11.937177 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7140ba9f-f03b-4d6a-8dc1-a246d260b20e None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:58:11.997089 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46f4a088-7cce-49fb-98b1-6ef32d04a19e None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:58:11.999369 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:58:12.021719 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:58:12.036646 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32dbb24f-d488-41a8-a49e-42656366d49a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:58:12.037562 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32dbb24f-d488-41a8-a49e-42656366d49a None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:58:12.037849 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32dbb24f-d488-41a8-a49e-42656366d49a None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:58:12.038125 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32dbb24f-d488-41a8-a49e-42656366d49a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:58:12.039194 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7140ba9f-f03b-4d6a-8dc1-a246d260b20e None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:58:12.073803 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04234ca6-1eed-4c77-8d22-67fd5333c18d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:58:12.074847 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04234ca6-1eed-4c77-8d22-67fd5333c18d None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:58:12.075123 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04234ca6-1eed-4c77-8d22-67fd5333c18d None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:58:12.075390 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04234ca6-1eed-4c77-8d22-67fd5333c18d None admin] PATH_INFO: `/v3/roles` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:58:12.076170 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04234ca6-1eed-4c77-8d22-67fd5333c18d None admin] RBAC: Adding query filter params (name=admin) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 21:58:12.076469 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04234ca6-1eed-4c77-8d22-67fd5333c18d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:58:12.077879 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04234ca6-1eed-4c77-8d22-67fd5333c18d None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:58:12.090327 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 8/15] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Jul 13 21:58:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 21:58:12.096197 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04234ca6-1eed-4c77-8d22-67fd5333c18d None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:58:12.106073 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32dbb24f-d488-41a8-a49e-42656366d49a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:58:12.107636 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32dbb24f-d488-41a8-a49e-42656366d49a None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:58:12.199639 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04234ca6-1eed-4c77-8d22-67fd5333c18d None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:58:12.207742 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 8/16] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Jul 13 21:58:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 316 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 13 21:58:12.214301 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32dbb24f-d488-41a8-a49e-42656366d49a None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:58:12.217103 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:58:12.235405 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2bd1ea9-8e81-4f8d-9783-5d3b214836ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:58:12.236294 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bd1ea9-8e81-4f8d-9783-5d3b214836ed None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:58:12.236568 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bd1ea9-8e81-4f8d-9783-5d3b214836ed None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:58:12.236854 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2bd1ea9-8e81-4f8d-9783-5d3b214836ed None admin] PATH_INFO: `/v3/users/cc99c93da5074cf5a48bc2aa9a92c5e7` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:58:12.238053 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2bd1ea9-8e81-4f8d-9783-5d3b214836ed None admin] RBAC: Authorizing `identity:get_user(user_id=cc99c93da5074cf5a48bc2aa9a92c5e7)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:58:12.239559 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2bd1ea9-8e81-4f8d-9783-5d3b214836ed None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:58:12.241192 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 9/17] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Jul 13 21:58:12 2022] GET /identity/v3/users/cc99c93da5074cf5a48bc2aa9a92c5e7 => generated 247 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 21:58:12.247110 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2bd1ea9-8e81-4f8d-9783-5d3b214836ed None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:58:12.261812 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:58:12.315100 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32dbb24f-d488-41a8-a49e-42656366d49a None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:58:12.346251 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2bd1ea9-8e81-4f8d-9783-5d3b214836ed None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:58:12.348320 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598002d4-6c9b-456d-a8e3-e9f715ebf622 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:58:12.349137 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598002d4-6c9b-456d-a8e3-e9f715ebf622 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:58:12.349417 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598002d4-6c9b-456d-a8e3-e9f715ebf622 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:58:12.349689 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-598002d4-6c9b-456d-a8e3-e9f715ebf622 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:58:12.369811 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598002d4-6c9b-456d-a8e3-e9f715ebf622 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:58:12.371299 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-598002d4-6c9b-456d-a8e3-e9f715ebf622 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:58:12.379545 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cdb4af0-38e1-4fa6-b707-b257555f5145 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:58:12.380353 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdb4af0-38e1-4fa6-b707-b257555f5145 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:58:12.380636 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdb4af0-38e1-4fa6-b707-b257555f5145 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:58:12.380911 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cdb4af0-38e1-4fa6-b707-b257555f5145 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:58:12.382105 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cdb4af0-38e1-4fa6-b707-b257555f5145 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:58:12.383606 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cdb4af0-38e1-4fa6-b707-b257555f5145 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:58:12.385267 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 10/18] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Jul 13 21:58:12 2022] GET /identity/v3/domains/default => generated 199 bytes in 139 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 21:58:12.391091 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cdb4af0-38e1-4fa6-b707-b257555f5145 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:58:12.391408 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application [None req-598002d4-6c9b-456d-a8e3-e9f715ebf622 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 21:58:12.391408 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 21:58:12.391408 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 21:58:12.391408 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 21:58:12.391408 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 21:58:12.391408 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 21:58:12.391408 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 21:58:12.391408 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 21:58:12.391408 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 21:58:12.391408 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 21:58:12.391408 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 21:58:12.391408 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 21:58:12.391408 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 21:58:12.391408 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 21:58:12.391408 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 21:58:12.392299 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 21:58:12.392299 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 21:58:12.392299 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 21:58:12.392299 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 21:58:12.392299 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 21:58:12.392299 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 21:58:12.392299 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 21:58:12.392299 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 21:58:12.392299 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application with Lock( Jul 13 21:58:12.392299 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 21:58:12.392299 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._enter() Jul 13 21:58:12.392299 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 21:58:12.392299 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 21:58:12.392299 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 21:58:12.392299 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.creator() Jul 13 21:58:12.392299 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 21:58:12.392299 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application created_value = creator( Jul 13 21:58:12.393109 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 21:58:12.393109 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 21:58:12.393109 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 21:58:12.393109 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Jul 13 21:58:12.393109 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 9/19] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Jul 13 21:58:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 13 21:58:12.398975 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598002d4-6c9b-456d-a8e3-e9f715ebf622 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:58:12.456273 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:58:12.472703 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:58:12.492090 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cdb4af0-38e1-4fa6-b707-b257555f5145 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:58:12.501390 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-598002d4-6c9b-456d-a8e3-e9f715ebf622 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:58:12.525967 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c991b1a-f55e-4b0f-a812-3e53518d0adf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:58:12.526660 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c991b1a-f55e-4b0f-a812-3e53518d0adf None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:58:12.526932 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c991b1a-f55e-4b0f-a812-3e53518d0adf None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:58:12.527226 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c991b1a-f55e-4b0f-a812-3e53518d0adf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:58:12.528059 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c991b1a-f55e-4b0f-a812-3e53518d0adf None admin] RBAC: Adding query filter params (role.id=44efba2579f342df8a0789c59d163ec3, scope.domain.id=default, user.id=cc99c93da5074cf5a48bc2aa9a92c5e7) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 21:58:12.528371 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c991b1a-f55e-4b0f-a812-3e53518d0adf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:58:12.529827 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c991b1a-f55e-4b0f-a812-3e53518d0adf None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:58:12.534746 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65604a88-ae1d-4ff9-85ad-b022949baa8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:58:12.535504 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65604a88-ae1d-4ff9-85ad-b022949baa8b None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:58:12.535793 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65604a88-ae1d-4ff9-85ad-b022949baa8b None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:58:12.536183 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65604a88-ae1d-4ff9-85ad-b022949baa8b None admin] PATH_INFO: `/v3/domains` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:58:12.537273 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 11/20] 10.222.0.52 () {56 vars in 1322 bytes} [Wed Jul 13 21:58:12 2022] GET /identity/v3/role_assignments?user.id=cc99c93da5074cf5a48bc2aa9a92c5e7&scope.domain.id=default&role.id=44efba2579f342df8a0789c59d163ec3 => generated 233 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 21:58:12.537372 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65604a88-ae1d-4ff9-85ad-b022949baa8b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 21:58:12.537372 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65604a88-ae1d-4ff9-85ad-b022949baa8b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:58:12.539430 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65604a88-ae1d-4ff9-85ad-b022949baa8b None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:58:12.559049 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 10/21] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Jul 13 21:58:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 21:58:14.922407 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c10902-79f6-45c2-8b44-7d386c97c7c5 None None] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:58:14.923300 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c10902-79f6-45c2-8b44-7d386c97c7c5 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:58:14.923815 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c10902-79f6-45c2-8b44-7d386c97c7c5 None None] PATH_INFO: `/` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:58:14.925339 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 12/22] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jul 13 21:58:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 21:58:14.930816 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bafe842-9832-4094-a1ec-b0a288d07d87 None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:58:14.931557 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bafe842-9832-4094-a1ec-b0a288d07d87 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:58:14.932029 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bafe842-9832-4094-a1ec-b0a288d07d87 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:58:14.956040 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-7bafe842-9832-4094-a1ec-b0a288d07d87 None None] MFA Rules not processed for user `cc99c93da5074cf5a48bc2aa9a92c5e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:58:14.957260 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-7bafe842-9832-4094-a1ec-b0a288d07d87 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:58:15.001624 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58474e87-cab4-427b-bd1a-0ef754d658f7 None None] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:58:15.002228 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58474e87-cab4-427b-bd1a-0ef754d658f7 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:58:15.002736 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-58474e87-cab4-427b-bd1a-0ef754d658f7 None None] PATH_INFO: `/` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:58:15.004326 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 13/23] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jul 13 21:58:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 13 21:58:15.011966 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae5a7b2-3919-4522-ba1e-e9d86cd3055c None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:58:15.012517 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae5a7b2-3919-4522-ba1e-e9d86cd3055c None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:58:15.012984 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae5a7b2-3919-4522-ba1e-e9d86cd3055c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:58:15.042514 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 11/24] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jul 13 21:58:14 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 13 21:58:15.043816 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-eae5a7b2-3919-4522-ba1e-e9d86cd3055c None None] MFA Rules not processed for user `cc99c93da5074cf5a48bc2aa9a92c5e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:58:15.045024 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-eae5a7b2-3919-4522-ba1e-e9d86cd3055c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:58:15.080624 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bafe842-9832-4094-a1ec-b0a288d07d87 None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:58:15.148652 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:58:15.148840 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 14/25] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jul 13 21:58:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 13 21:58:15.187144 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae5a7b2-3919-4522-ba1e-e9d86cd3055c None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:58:15.213666 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bafe842-9832-4094-a1ec-b0a288d07d87 None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:58:15.274908 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:58:15.295782 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-599e0fcb-f765-4ccb-b6a2-9dbd7f776dbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:58:15.296603 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599e0fcb-f765-4ccb-b6a2-9dbd7f776dbd None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:58:15.296880 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599e0fcb-f765-4ccb-b6a2-9dbd7f776dbd None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:58:15.297145 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-599e0fcb-f765-4ccb-b6a2-9dbd7f776dbd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:58:15.317031 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599e0fcb-f765-4ccb-b6a2-9dbd7f776dbd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:58:15.318533 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-599e0fcb-f765-4ccb-b6a2-9dbd7f776dbd None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:58:15.325175 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae5a7b2-3919-4522-ba1e-e9d86cd3055c None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:58:15.356974 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application [None req-599e0fcb-f765-4ccb-b6a2-9dbd7f776dbd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 21:58:15.356974 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 21:58:15.356974 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 21:58:15.356974 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 21:58:15.356974 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 21:58:15.356974 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 21:58:15.356974 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 21:58:15.356974 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 21:58:15.356974 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 21:58:15.356974 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 21:58:15.356974 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 21:58:15.356974 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 21:58:15.356974 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 21:58:15.356974 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 21:58:15.356974 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 21:58:15.357849 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 21:58:15.357849 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 21:58:15.357849 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 21:58:15.357849 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 21:58:15.357849 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 21:58:15.357849 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 21:58:15.357849 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 21:58:15.357849 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 21:58:15.357849 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application with Lock( Jul 13 21:58:15.357849 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 21:58:15.357849 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._enter() Jul 13 21:58:15.357849 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 21:58:15.357849 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 21:58:15.357849 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 21:58:15.357849 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.creator() Jul 13 21:58:15.357849 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 21:58:15.357849 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application created_value = creator( Jul 13 21:58:15.358675 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 21:58:15.358675 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 21:58:15.358675 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 21:58:15.358675 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Jul 13 21:58:15.358675 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 12/26] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Jul 13 21:58:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 278 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 21:58:15.367984 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e13f845-de2d-4c77-8a3f-b579277ec67a None None] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:58:15.368260 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e13f845-de2d-4c77-8a3f-b579277ec67a None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:58:15.368530 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e13f845-de2d-4c77-8a3f-b579277ec67a None None] PATH_INFO: `/` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:58:15.370283 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 13/27] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jul 13 21:58:15 2022] GET /identity => generated 270 bytes in 11 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 21:58:15.370688 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e13f845-de2d-4c77-8a3f-b579277ec67a None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:58:15.374384 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-554b997e-52d8-484c-bf07-814cf9731ab4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:58:15.375186 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554b997e-52d8-484c-bf07-814cf9731ab4 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:58:15.375466 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554b997e-52d8-484c-bf07-814cf9731ab4 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:58:15.375737 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-554b997e-52d8-484c-bf07-814cf9731ab4 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:58:15.397085 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-554b997e-52d8-484c-bf07-814cf9731ab4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 21:58:15.413107 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application [None req-554b997e-52d8-484c-bf07-814cf9731ab4 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 13 21:58:15.413107 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 21:58:15.413107 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 21:58:15.413107 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 21:58:15.413107 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 21:58:15.413107 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 21:58:15.413107 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 21:58:15.413107 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 21:58:15.413107 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 21:58:15.413107 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 21:58:15.413107 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 21:58:15.413107 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 21:58:15.413107 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 21:58:15.413107 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 21:58:15.413107 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 21:58:15.413107 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 21:58:15.414055 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 21:58:15.414055 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 21:58:15.414055 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 21:58:15.414055 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 21:58:15.414055 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 21:58:15.414055 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 21:58:15.414055 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 21:58:15.414055 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 21:58:15.414055 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 21:58:15.414055 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 21:58:15.414055 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 21:58:15.414055 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application with Lock( Jul 13 21:58:15.414055 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 21:58:15.414055 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._enter() Jul 13 21:58:15.414055 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 21:58:15.414055 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 21:58:15.414842 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 21:58:15.414842 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.creator() Jul 13 21:58:15.414842 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 21:58:15.414842 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application created_value = creator( Jul 13 21:58:15.414842 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 21:58:15.414842 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 21:58:15.414842 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 21:58:15.414842 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 21:58:15.414842 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 21:58:15.414842 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 21:58:15.414842 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 13 21:58:15.414842 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Jul 13 21:58:15.414842 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 15/28] 10.222.0.52 () {56 vars in 1096 bytes} [Wed Jul 13 21:58:15 2022] GET /identity/v3/roles/admin => generated 83 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 13 21:58:15.422510 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2989d73-276e-4371-81c3-cf0738c50864 None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:58:15.423050 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2989d73-276e-4371-81c3-cf0738c50864 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:58:15.423471 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2989d73-276e-4371-81c3-cf0738c50864 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:58:15.459641 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-f2989d73-276e-4371-81c3-cf0738c50864 None None] MFA Rules not processed for user `cc99c93da5074cf5a48bc2aa9a92c5e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:58:15.460784 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-f2989d73-276e-4371-81c3-cf0738c50864 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:58:15.490892 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:58:15.496870 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e13f845-de2d-4c77-8a3f-b579277ec67a None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:58:15.566509 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7106297-7cb6-41f9-a37e-4e50e76ff8b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:58:15.566509 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7106297-7cb6-41f9-a37e-4e50e76ff8b0 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:58:15.566509 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7106297-7cb6-41f9-a37e-4e50e76ff8b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:58:15.566509 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7106297-7cb6-41f9-a37e-4e50e76ff8b0 None admin] PATH_INFO: `/v3/domains` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:58:15.567159 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7106297-7cb6-41f9-a37e-4e50e76ff8b0 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 21:58:15.567475 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7106297-7cb6-41f9-a37e-4e50e76ff8b0 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:58:15.570351 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7106297-7cb6-41f9-a37e-4e50e76ff8b0 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:58:15.587933 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 14/29] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Jul 13 21:58:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 21:58:15.589052 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7106297-7cb6-41f9-a37e-4e50e76ff8b0 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:58:15.623583 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 16/30] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jul 13 21:58:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 209 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 13 21:58:15.624419 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2989d73-276e-4371-81c3-cf0738c50864 None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:58:15.696127 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7106297-7cb6-41f9-a37e-4e50e76ff8b0 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:58:15.709535 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:58:15.734573 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4543c9b3-436e-461d-81ac-729728aa6118 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:58:15.735477 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4543c9b3-436e-461d-81ac-729728aa6118 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:58:15.735909 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4543c9b3-436e-461d-81ac-729728aa6118 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:58:15.736340 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4543c9b3-436e-461d-81ac-729728aa6118 None admin] PATH_INFO: `/v3/roles` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:58:15.737351 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4543c9b3-436e-461d-81ac-729728aa6118 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 21:58:15.737796 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4543c9b3-436e-461d-81ac-729728aa6118 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:58:15.739378 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4543c9b3-436e-461d-81ac-729728aa6118 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:58:15.749749 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2989d73-276e-4371-81c3-cf0738c50864 None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:58:15.753085 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 15/31] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Jul 13 21:58:15 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 21:58:15.754845 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80d707d-7a3b-4956-b8a1-6d248626d16a None None] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:58:15.755327 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80d707d-7a3b-4956-b8a1-6d248626d16a None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:58:15.755851 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80d707d-7a3b-4956-b8a1-6d248626d16a None None] PATH_INFO: `/` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:58:15.757465 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 16/32] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jul 13 21:58:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 21:58:15.758453 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80d707d-7a3b-4956-b8a1-6d248626d16a None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:58:15.785872 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:58:15.792751 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39428fcb-9b85-49a9-b8d3-da05272143d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:58:15.793615 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39428fcb-9b85-49a9-b8d3-da05272143d3 None admin] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:58:15.794034 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39428fcb-9b85-49a9-b8d3-da05272143d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:58:15.794688 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39428fcb-9b85-49a9-b8d3-da05272143d3 None admin] PATH_INFO: `/v3/projects` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:58:15.796372 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39428fcb-9b85-49a9-b8d3-da05272143d3 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:58:15.798166 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39428fcb-9b85-49a9-b8d3-da05272143d3 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:58:15.872476 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80d707d-7a3b-4956-b8a1-6d248626d16a None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:58:15.907963 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04067100-f004-4a38-a9f9-1d3677df63e6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:58:15.909080 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04067100-f004-4a38-a9f9-1d3677df63e6 None admin] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:58:15.909554 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04067100-f004-4a38-a9f9-1d3677df63e6 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:58:15.910037 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-04067100-f004-4a38-a9f9-1d3677df63e6 None admin] PATH_INFO: `/v3/roles` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:58:15.911021 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04067100-f004-4a38-a9f9-1d3677df63e6 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:58:15.912801 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-04067100-f004-4a38-a9f9-1d3677df63e6 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:58:15.942850 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 17/33] 10.222.0.52 () {60 vars in 1141 bytes} [Wed Jul 13 21:58:15 2022] POST /identity/v3/projects => generated 301 bytes in 319 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 21:58:15.944506 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39428fcb-9b85-49a9-b8d3-da05272143d3 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:58:16.052088 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39428fcb-9b85-49a9-b8d3-da05272143d3 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:58:16.070181 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:58:16.089781 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2a7347-97a4-4ffc-89af-6f293a03f558 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:58:16.091109 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2a7347-97a4-4ffc-89af-6f293a03f558 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:58:16.091584 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2a7347-97a4-4ffc-89af-6f293a03f558 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:58:16.092060 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2a7347-97a4-4ffc-89af-6f293a03f558 None admin] PATH_INFO: `/v3/users/cc99c93da5074cf5a48bc2aa9a92c5e7` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:58:16.093569 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2a7347-97a4-4ffc-89af-6f293a03f558 None admin] RBAC: Authorizing `identity:get_user(user_id=cc99c93da5074cf5a48bc2aa9a92c5e7)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:58:16.095394 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2a7347-97a4-4ffc-89af-6f293a03f558 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:58:16.097421 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 18/34] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Jul 13 21:58:15 2022] GET /identity/v3/users/cc99c93da5074cf5a48bc2aa9a92c5e7 => generated 247 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 21:58:16.099255 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27d5976-578c-41b0-a471-907315ba40f0 None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:58:16.099775 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27d5976-578c-41b0-a471-907315ba40f0 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:58:16.100236 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a27d5976-578c-41b0-a471-907315ba40f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:58:16.107777 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 17/35] 10.222.0.52 () {60 vars in 1131 bytes} [Wed Jul 13 21:58:15 2022] POST /identity/v3/roles => generated 225 bytes in 350 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 21:58:16.115860 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04067100-f004-4a38-a9f9-1d3677df63e6 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:58:16.128511 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-a27d5976-578c-41b0-a471-907315ba40f0 None None] MFA Rules not processed for user `cc99c93da5074cf5a48bc2aa9a92c5e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:58:16.134893 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-a27d5976-578c-41b0-a471-907315ba40f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:58:16.260489 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:58:16.283075 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-04067100-f004-4a38-a9f9-1d3677df63e6 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:58:16.286466 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 19/36] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jul 13 21:58:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 13 21:58:16.288778 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabcc63a-2e2a-44f8-b62c-7399ad4794ec None None] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:58:16.289086 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabcc63a-2e2a-44f8-b62c-7399ad4794ec None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:58:16.289367 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aabcc63a-2e2a-44f8-b62c-7399ad4794ec None None] PATH_INFO: `/` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:58:16.291084 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 20/37] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jul 13 21:58:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 21:58:16.300176 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd341ec-1db8-4ed9-88e5-72286af097a9 None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:58:16.300502 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd341ec-1db8-4ed9-88e5-72286af097a9 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:58:16.300742 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efd341ec-1db8-4ed9-88e5-72286af097a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:58:16.324139 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95c581e-b897-418a-baae-05653203623a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:58:16.325230 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95c581e-b897-418a-baae-05653203623a None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:58:16.325540 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95c581e-b897-418a-baae-05653203623a None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:58:16.325867 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95c581e-b897-418a-baae-05653203623a None admin] PATH_INFO: `/v3/domains/default` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:58:16.327679 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95c581e-b897-418a-baae-05653203623a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:58:16.329376 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e95c581e-b897-418a-baae-05653203623a None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:58:16.331560 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 18/38] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Jul 13 21:58:16 2022] GET /identity/v3/domains/default => generated 199 bytes in 216 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 21:58:16.332392 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95c581e-b897-418a-baae-05653203623a None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:58:16.347599 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-efd341ec-1db8-4ed9-88e5-72286af097a9 None None] MFA Rules not processed for user `cc99c93da5074cf5a48bc2aa9a92c5e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:58:16.348831 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-efd341ec-1db8-4ed9-88e5-72286af097a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:58:16.454697 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e95c581e-b897-418a-baae-05653203623a None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:58:16.459641 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 21/39] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jul 13 21:58:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 13 21:58:16.460557 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd341ec-1db8-4ed9-88e5-72286af097a9 None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:58:16.499176 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2762abc5-8eaf-409e-a687-f5df633d8c06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:58:16.499852 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2762abc5-8eaf-409e-a687-f5df633d8c06 None admin] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:58:16.500134 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2762abc5-8eaf-409e-a687-f5df633d8c06 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:58:16.500407 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2762abc5-8eaf-409e-a687-f5df633d8c06 None admin] PATH_INFO: `/v3/roles` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:58:16.501177 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2762abc5-8eaf-409e-a687-f5df633d8c06 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:58:16.502625 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2762abc5-8eaf-409e-a687-f5df633d8c06 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:58:16.517836 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 19/40] 10.222.0.52 () {60 vars in 1131 bytes} [Wed Jul 13 21:58:16 2022] POST /identity/v3/roles => generated 219 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 21:58:16.518751 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2762abc5-8eaf-409e-a687-f5df633d8c06 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:58:16.532714 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:58:16.622245 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:58:16.648546 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2762abc5-8eaf-409e-a687-f5df633d8c06 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:58:16.687800 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e55b9a4-113a-48b6-a1c9-c885a54642d3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:58:16.689299 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e55b9a4-113a-48b6-a1c9-c885a54642d3 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:58:16.689299 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e55b9a4-113a-48b6-a1c9-c885a54642d3 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:58:16.689299 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e55b9a4-113a-48b6-a1c9-c885a54642d3 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:58:16.689544 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efd341ec-1db8-4ed9-88e5-72286af097a9 None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:58:16.690769 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e55b9a4-113a-48b6-a1c9-c885a54642d3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:58:16.692527 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e55b9a4-113a-48b6-a1c9-c885a54642d3 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:58:16.699200 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 20/41] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Jul 13 21:58:16 2022] GET /identity/v3/domains/default => generated 199 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 21:58:16.699200 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bb72cb-b3cc-49be-8c94-6dacb368a9d4 None None] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:58:16.699200 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bb72cb-b3cc-49be-8c94-6dacb368a9d4 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:58:16.699200 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9bb72cb-b3cc-49be-8c94-6dacb368a9d4 None None] PATH_INFO: `/` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:58:16.699200 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 21/42] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jul 13 21:58:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 21:58:16.699929 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9bb72cb-b3cc-49be-8c94-6dacb368a9d4 None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:58:16.735680 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1f17b3-60d8-4858-afde-531f2e946419 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:58:16.736580 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1f17b3-60d8-4858-afde-531f2e946419 None admin] REQUEST_METHOD: `PUT` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:58:16.736859 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1f17b3-60d8-4858-afde-531f2e946419 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:58:16.737139 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1f17b3-60d8-4858-afde-531f2e946419 None admin] PATH_INFO: `/v3/domains/default/users/cc99c93da5074cf5a48bc2aa9a92c5e7/roles/44efba2579f342df8a0789c59d163ec3` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:58:16.739159 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad1f17b3-60d8-4858-afde-531f2e946419 None admin] RBAC: Authorizing `identity:create_grant(domain_id=default, user_id=cc99c93da5074cf5a48bc2aa9a92c5e7, role_id=44efba2579f342df8a0789c59d163ec3)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:58:16.740646 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad1f17b3-60d8-4858-afde-531f2e946419 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:58:16.756148 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 22/43] 10.222.0.52 () {58 vars in 1361 bytes} [Wed Jul 13 21:58:16 2022] PUT /identity/v3/domains/default/users/cc99c93da5074cf5a48bc2aa9a92c5e7/roles/44efba2579f342df8a0789c59d163ec3 => generated 0 bytes in 296 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 21:58:16.757478 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a64b31f-c3aa-419f-8fd7-7bf97342a280 None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:58:16.757772 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a64b31f-c3aa-419f-8fd7-7bf97342a280 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:58:16.758053 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a64b31f-c3aa-419f-8fd7-7bf97342a280 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:58:16.776720 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-3a64b31f-c3aa-419f-8fd7-7bf97342a280 None None] MFA Rules not processed for user `cc99c93da5074cf5a48bc2aa9a92c5e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:58:16.777979 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-3a64b31f-c3aa-419f-8fd7-7bf97342a280 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:58:16.798407 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:58:16.831047 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9bb72cb-b3cc-49be-8c94-6dacb368a9d4 None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:58:16.852552 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 23/44] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jul 13 21:58:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 13 21:58:16.877346 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a64b31f-c3aa-419f-8fd7-7bf97342a280 None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:58:16.878873 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0368e74-44b6-4d88-b8d7-60f34d7795e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:58:16.879841 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0368e74-44b6-4d88-b8d7-60f34d7795e3 None admin] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:58:16.880382 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0368e74-44b6-4d88-b8d7-60f34d7795e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:58:16.880660 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0368e74-44b6-4d88-b8d7-60f34d7795e3 None admin] PATH_INFO: `/v3/projects` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:58:16.881525 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0368e74-44b6-4d88-b8d7-60f34d7795e3 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:58:16.883125 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0368e74-44b6-4d88-b8d7-60f34d7795e3 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:58:16.916454 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 22/45] 10.222.0.52 () {60 vars in 1141 bytes} [Wed Jul 13 21:58:16 2022] POST /identity/v3/projects => generated 312 bytes in 216 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 21:58:16.917107 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:58:16.997277 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a64b31f-c3aa-419f-8fd7-7bf97342a280 None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:58:17.037182 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77efaf1b-d298-4b31-a81d-5170c07c74b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:58:17.038275 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77efaf1b-d298-4b31-a81d-5170c07c74b3 None admin] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:58:17.038789 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77efaf1b-d298-4b31-a81d-5170c07c74b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:58:17.039256 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77efaf1b-d298-4b31-a81d-5170c07c74b3 None admin] PATH_INFO: `/v3/roles` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:58:17.040229 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77efaf1b-d298-4b31-a81d-5170c07c74b3 None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:58:17.042267 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77efaf1b-d298-4b31-a81d-5170c07c74b3 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:58:17.060680 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 24/46] 10.222.0.52 () {60 vars in 1131 bytes} [Wed Jul 13 21:58:16 2022] POST /identity/v3/roles => generated 223 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 21:58:18.812201 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43c9b0d-8167-43cd-94a3-d5f9ed30196c None None] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:58:18.812720 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43c9b0d-8167-43cd-94a3-d5f9ed30196c None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:58:18.812720 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43c9b0d-8167-43cd-94a3-d5f9ed30196c None None] PATH_INFO: `/` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:58:18.814039 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 23/47] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jul 13 21:58:18 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 21:58:18.819676 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452dc403-b527-431a-b155-1e5c004f1f6f None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:58:18.820126 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452dc403-b527-431a-b155-1e5c004f1f6f None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:58:18.820648 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-452dc403-b527-431a-b155-1e5c004f1f6f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:58:18.843103 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-452dc403-b527-431a-b155-1e5c004f1f6f None None] MFA Rules not processed for user `cc99c93da5074cf5a48bc2aa9a92c5e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:58:18.844321 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-452dc403-b527-431a-b155-1e5c004f1f6f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:58:18.928588 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 25/48] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jul 13 21:58:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 13 21:58:18.952957 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43c9b0d-8167-43cd-94a3-d5f9ed30196c None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:58:18.983302 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da4221b-09aa-4895-8e7a-196f6a87f8a4 None None] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:58:18.983594 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da4221b-09aa-4895-8e7a-196f6a87f8a4 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:58:18.983878 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da4221b-09aa-4895-8e7a-196f6a87f8a4 None None] PATH_INFO: `/` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:58:18.985306 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 26/49] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jul 13 21:58:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 21:58:18.990467 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa57022-1b41-4576-8eca-b4c1ee56389e None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:58:18.990791 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa57022-1b41-4576-8eca-b4c1ee56389e None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:58:18.991074 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfa57022-1b41-4576-8eca-b4c1ee56389e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:58:19.008192 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-dfa57022-1b41-4576-8eca-b4c1ee56389e None None] MFA Rules not processed for user `cc99c93da5074cf5a48bc2aa9a92c5e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:58:19.010228 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-dfa57022-1b41-4576-8eca-b4c1ee56389e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:58:19.031366 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:58:19.076322 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43c9b0d-8167-43cd-94a3-d5f9ed30196c None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:58:19.089686 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 27/50] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jul 13 21:58:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 13 21:58:19.108084 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:58:19.112884 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa57022-1b41-4576-8eca-b4c1ee56389e None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:58:19.119426 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9484b6ab-7098-4482-864c-6edccd282695 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:58:19.120492 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9484b6ab-7098-4482-864c-6edccd282695 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:58:19.121006 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9484b6ab-7098-4482-864c-6edccd282695 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:58:19.121296 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9484b6ab-7098-4482-864c-6edccd282695 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:58:19.136000 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9484b6ab-7098-4482-864c-6edccd282695 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 21:58:19.146187 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application [None req-9484b6ab-7098-4482-864c-6edccd282695 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 13 21:58:19.146187 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 21:58:19.146187 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 21:58:19.146187 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 21:58:19.146187 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 21:58:19.146187 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 21:58:19.146187 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 21:58:19.146187 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 21:58:19.146187 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 21:58:19.146187 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 21:58:19.146187 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 21:58:19.146187 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 21:58:19.146187 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 21:58:19.146187 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 21:58:19.146187 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 21:58:19.146187 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 21:58:19.147106 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 21:58:19.147106 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 21:58:19.147106 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 21:58:19.147106 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 21:58:19.147106 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 21:58:19.147106 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 21:58:19.147106 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 21:58:19.147106 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 21:58:19.147106 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 21:58:19.147106 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 21:58:19.147106 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 21:58:19.147106 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application with Lock( Jul 13 21:58:19.147106 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 21:58:19.147106 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._enter() Jul 13 21:58:19.147106 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 21:58:19.147106 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 21:58:19.148094 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 21:58:19.148094 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.creator() Jul 13 21:58:19.148094 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 21:58:19.148094 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application created_value = creator( Jul 13 21:58:19.148094 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 21:58:19.148094 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 21:58:19.148094 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 21:58:19.148094 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 21:58:19.148094 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 21:58:19.148094 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 21:58:19.148094 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 13 21:58:19.148094 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Jul 13 21:58:19.148094 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 24/51] 10.222.0.52 () {56 vars in 1096 bytes} [Wed Jul 13 21:58:18 2022] GET /identity/v3/roles/admin => generated 83 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 21:58:19.153921 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9484b6ab-7098-4482-864c-6edccd282695 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:58:19.227985 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfa57022-1b41-4576-8eca-b4c1ee56389e None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:58:19.268749 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569733dc-d81d-4a10-a0f1-12a061eefb2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:58:19.269628 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569733dc-d81d-4a10-a0f1-12a061eefb2a None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:58:19.269911 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569733dc-d81d-4a10-a0f1-12a061eefb2a None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:58:19.270005 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9484b6ab-7098-4482-864c-6edccd282695 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:58:19.270186 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-569733dc-d81d-4a10-a0f1-12a061eefb2a None admin] PATH_INFO: `/v3/domains/default` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:58:19.273128 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569733dc-d81d-4a10-a0f1-12a061eefb2a None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:58:19.274524 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-569733dc-d81d-4a10-a0f1-12a061eefb2a None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:58:19.276904 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 28/52] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Jul 13 21:58:19 2022] GET /identity/v3/domains/default => generated 199 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 21:58:19.287892 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569733dc-d81d-4a10-a0f1-12a061eefb2a None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:58:19.312438 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f9f9a3-543e-47bb-801f-2b39886a1b74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:58:19.313464 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f9f9a3-543e-47bb-801f-2b39886a1b74 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:58:19.313943 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f9f9a3-543e-47bb-801f-2b39886a1b74 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:58:19.314407 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8f9f9a3-543e-47bb-801f-2b39886a1b74 None admin] PATH_INFO: `/v3/roles` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:58:19.315756 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f9f9a3-543e-47bb-801f-2b39886a1b74 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 21:58:19.316273 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f9f9a3-543e-47bb-801f-2b39886a1b74 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:58:19.318726 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8f9f9a3-543e-47bb-801f-2b39886a1b74 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:58:19.319636 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:58:19.332119 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 25/53] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Jul 13 21:58:19 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 21:58:19.337769 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f9f9a3-543e-47bb-801f-2b39886a1b74 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:58:19.391416 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:58:19.407580 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-569733dc-d81d-4a10-a0f1-12a061eefb2a None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:58:19.441999 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcc7411-b36b-407a-aba4-fdf6ece7cd21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:58:19.442783 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcc7411-b36b-407a-aba4-fdf6ece7cd21 None admin] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:58:19.443105 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcc7411-b36b-407a-aba4-fdf6ece7cd21 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:58:19.443191 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8f9f9a3-543e-47bb-801f-2b39886a1b74 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:58:19.443386 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fcc7411-b36b-407a-aba4-fdf6ece7cd21 None admin] PATH_INFO: `/v3/projects` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:58:19.444187 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fcc7411-b36b-407a-aba4-fdf6ece7cd21 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:58:19.445656 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fcc7411-b36b-407a-aba4-fdf6ece7cd21 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:58:19.474111 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 29/54] 10.222.0.52 () {60 vars in 1140 bytes} [Wed Jul 13 21:58:19 2022] POST /identity/v3/projects => generated 298 bytes in 191 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 21:58:19.477900 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3a70d4-1e8e-4eed-9ad4-a75216e690dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:58:19.478986 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3a70d4-1e8e-4eed-9ad4-a75216e690dc None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:58:19.479274 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3a70d4-1e8e-4eed-9ad4-a75216e690dc None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:58:19.479553 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd3a70d4-1e8e-4eed-9ad4-a75216e690dc None admin] PATH_INFO: `/v3/users/cc99c93da5074cf5a48bc2aa9a92c5e7` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:58:19.481131 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd3a70d4-1e8e-4eed-9ad4-a75216e690dc None admin] RBAC: Authorizing `identity:get_user(user_id=cc99c93da5074cf5a48bc2aa9a92c5e7)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:58:19.482681 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd3a70d4-1e8e-4eed-9ad4-a75216e690dc None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:58:19.484463 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 26/55] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Jul 13 21:58:19 2022] GET /identity/v3/users/cc99c93da5074cf5a48bc2aa9a92c5e7 => generated 247 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 21:58:19.490321 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcc7411-b36b-407a-aba4-fdf6ece7cd21 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:58:19.576251 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:58:19.605332 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fcc7411-b36b-407a-aba4-fdf6ece7cd21 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:58:19.642504 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c36fb71-107e-4cde-8375-9302d52bb068 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:58:19.643403 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c36fb71-107e-4cde-8375-9302d52bb068 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:58:19.643684 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c36fb71-107e-4cde-8375-9302d52bb068 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:58:19.643955 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c36fb71-107e-4cde-8375-9302d52bb068 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:58:19.645273 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c36fb71-107e-4cde-8375-9302d52bb068 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:58:19.646817 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c36fb71-107e-4cde-8375-9302d52bb068 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:58:19.648425 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 30/56] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Jul 13 21:58:19 2022] GET /identity/v3/domains/default => generated 199 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 21:58:19.654402 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3a70d4-1e8e-4eed-9ad4-a75216e690dc None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:58:19.868420 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd3a70d4-1e8e-4eed-9ad4-a75216e690dc None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:58:19.885611 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:58:19.903441 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b2acfb7-123e-4253-9f23-e502329b65c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:58:19.904106 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2acfb7-123e-4253-9f23-e502329b65c6 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:58:19.904384 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2acfb7-123e-4253-9f23-e502329b65c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:58:19.904656 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b2acfb7-123e-4253-9f23-e502329b65c6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:58:19.905499 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2acfb7-123e-4253-9f23-e502329b65c6 None admin] RBAC: Adding query filter params (role.id=44efba2579f342df8a0789c59d163ec3, scope.domain.id=default, user.id=cc99c93da5074cf5a48bc2aa9a92c5e7) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 21:58:19.905806 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2acfb7-123e-4253-9f23-e502329b65c6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:58:19.907295 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b2acfb7-123e-4253-9f23-e502329b65c6 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:58:19.916623 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 27/57] 10.222.0.52 () {56 vars in 1322 bytes} [Wed Jul 13 21:58:19 2022] GET /identity/v3/role_assignments?user.id=cc99c93da5074cf5a48bc2aa9a92c5e7&scope.domain.id=default&role.id=44efba2579f342df8a0789c59d163ec3 => generated 530 bytes in 263 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 21:58:21.559528 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1751ee1b-a0cd-45fb-b90a-38dc44198128 None None] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:58:21.559957 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1751ee1b-a0cd-45fb-b90a-38dc44198128 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:58:21.560069 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1751ee1b-a0cd-45fb-b90a-38dc44198128 None None] PATH_INFO: `/` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:58:21.561498 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 31/58] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jul 13 21:58:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 21:58:21.566771 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d3613d-6e9e-4688-8940-d6ecc95170a7 None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:58:21.567185 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d3613d-6e9e-4688-8940-d6ecc95170a7 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:58:21.567323 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9d3613d-6e9e-4688-8940-d6ecc95170a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:58:21.586133 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-a9d3613d-6e9e-4688-8940-d6ecc95170a7 None None] MFA Rules not processed for user `cc99c93da5074cf5a48bc2aa9a92c5e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:58:21.587714 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-a9d3613d-6e9e-4688-8940-d6ecc95170a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:58:21.656490 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 28/59] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jul 13 21:58:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 13 21:58:21.681482 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1751ee1b-a0cd-45fb-b90a-38dc44198128 None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:58:21.796515 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1751ee1b-a0cd-45fb-b90a-38dc44198128 None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:58:21.832341 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02f06f37-a1e8-407b-9e93-ac4dd2471773 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:58:21.833073 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f06f37-a1e8-407b-9e93-ac4dd2471773 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:58:21.833354 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f06f37-a1e8-407b-9e93-ac4dd2471773 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:58:21.833626 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02f06f37-a1e8-407b-9e93-ac4dd2471773 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:58:21.834956 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f06f37-a1e8-407b-9e93-ac4dd2471773 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:58:21.836418 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02f06f37-a1e8-407b-9e93-ac4dd2471773 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:58:21.838178 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 32/60] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Jul 13 21:58:21 2022] GET /identity/v3/domains/default => generated 199 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 21:58:21.843832 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d3613d-6e9e-4688-8940-d6ecc95170a7 None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:58:21.961916 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9d3613d-6e9e-4688-8940-d6ecc95170a7 None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:58:21.975212 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:58:21.998555 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84859fb-e897-4208-97a8-d9228605aba3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:58:21.999254 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84859fb-e897-4208-97a8-d9228605aba3 None admin] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:58:21.999532 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84859fb-e897-4208-97a8-d9228605aba3 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:58:21.999810 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84859fb-e897-4208-97a8-d9228605aba3 None admin] PATH_INFO: `/v3/users` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:58:22.000535 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d84859fb-e897-4208-97a8-d9228605aba3 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:58:22.002003 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d84859fb-e897-4208-97a8-d9228605aba3 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:58:22.029668 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 29/61] 10.222.0.52 () {60 vars in 1132 bytes} [Wed Jul 13 21:58:21 2022] POST /identity/v3/users => generated 275 bytes in 186 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 21:58:24.964776 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c3407a-9e2a-4b18-8d74-54744edc2f13 None None] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:58:24.965990 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c3407a-9e2a-4b18-8d74-54744edc2f13 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:58:24.966508 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79c3407a-9e2a-4b18-8d74-54744edc2f13 None None] PATH_INFO: `/` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:58:24.968394 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 33/62] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jul 13 21:58:24 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 21:58:24.973492 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df147c7e-cf9f-46f8-9cb8-4df37892724b None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:58:24.974309 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df147c7e-cf9f-46f8-9cb8-4df37892724b None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:58:24.974823 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df147c7e-cf9f-46f8-9cb8-4df37892724b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:58:24.999549 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-df147c7e-cf9f-46f8-9cb8-4df37892724b None None] MFA Rules not processed for user `cc99c93da5074cf5a48bc2aa9a92c5e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:58:25.000777 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-df147c7e-cf9f-46f8-9cb8-4df37892724b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:58:25.092824 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 30/63] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jul 13 21:58:24 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 13 21:58:25.140040 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c3407a-9e2a-4b18-8d74-54744edc2f13 None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:58:25.158085 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:58:25.270259 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79c3407a-9e2a-4b18-8d74-54744edc2f13 None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:58:25.304643 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03cfa215-11ff-48b3-b32a-930afa8dbd85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:58:25.305463 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cfa215-11ff-48b3-b32a-930afa8dbd85 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:58:25.305753 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cfa215-11ff-48b3-b32a-930afa8dbd85 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:58:25.306032 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cfa215-11ff-48b3-b32a-930afa8dbd85 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:58:25.316847 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03cfa215-11ff-48b3-b32a-930afa8dbd85 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 21:58:25.325085 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application [None req-03cfa215-11ff-48b3-b32a-930afa8dbd85 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 13 21:58:25.325085 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 21:58:25.325085 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 21:58:25.325085 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 21:58:25.325085 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 21:58:25.325085 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 21:58:25.325085 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 21:58:25.325085 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 21:58:25.325085 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 21:58:25.325085 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 21:58:25.325085 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 21:58:25.325085 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 21:58:25.325085 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 21:58:25.325085 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 21:58:25.325085 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 21:58:25.325085 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 21:58:25.325934 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 21:58:25.325934 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 21:58:25.325934 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 21:58:25.325934 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 21:58:25.325934 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 21:58:25.325934 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 21:58:25.325934 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 21:58:25.325934 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 21:58:25.325934 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 21:58:25.325934 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 21:58:25.325934 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 21:58:25.325934 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application with Lock( Jul 13 21:58:25.325934 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 21:58:25.325934 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._enter() Jul 13 21:58:25.325934 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 21:58:25.325934 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 21:58:25.326725 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 21:58:25.326725 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.creator() Jul 13 21:58:25.326725 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 21:58:25.326725 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application created_value = creator( Jul 13 21:58:25.326725 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 21:58:25.326725 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 21:58:25.326725 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 21:58:25.326725 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 21:58:25.326725 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 21:58:25.326725 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 21:58:25.326725 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 13 21:58:25.326725 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Jul 13 21:58:25.327301 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 34/64] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Jul 13 21:58:25 2022] GET /identity/v3/roles/member => generated 84 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 21:58:25.335603 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df147c7e-cf9f-46f8-9cb8-4df37892724b None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:58:25.450116 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:58:25.454783 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df147c7e-cf9f-46f8-9cb8-4df37892724b None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:58:25.492261 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89ed2c26-5468-4455-99a5-f75e46568e4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:58:25.492955 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ed2c26-5468-4455-99a5-f75e46568e4f None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:58:25.493241 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ed2c26-5468-4455-99a5-f75e46568e4f None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:58:25.493521 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89ed2c26-5468-4455-99a5-f75e46568e4f None admin] PATH_INFO: `/v3/roles` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:58:25.494292 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ed2c26-5468-4455-99a5-f75e46568e4f None admin] RBAC: Adding query filter params (name=member) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 21:58:25.494635 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ed2c26-5468-4455-99a5-f75e46568e4f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:58:25.496319 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89ed2c26-5468-4455-99a5-f75e46568e4f None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:58:25.505185 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 31/65] 10.222.0.52 () {56 vars in 1101 bytes} [Wed Jul 13 21:58:25 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 21:58:25.511170 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03cfa215-11ff-48b3-b32a-930afa8dbd85 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:58:25.540658 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5bdd17-b5e3-474e-9609-bfbcf41c16b1 None None] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:58:25.541186 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5bdd17-b5e3-474e-9609-bfbcf41c16b1 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:58:25.541607 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f5bdd17-b5e3-474e-9609-bfbcf41c16b1 None None] PATH_INFO: `/` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:58:25.543468 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 32/66] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jul 13 21:58:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 21:58:25.550964 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67742f2-be1c-47eb-99dd-25d6afc3b477 None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:58:25.552035 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67742f2-be1c-47eb-99dd-25d6afc3b477 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:58:25.552441 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e67742f2-be1c-47eb-99dd-25d6afc3b477 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:58:25.570638 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-e67742f2-be1c-47eb-99dd-25d6afc3b477 None None] MFA Rules not processed for user `cc99c93da5074cf5a48bc2aa9a92c5e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:58:25.572081 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-e67742f2-be1c-47eb-99dd-25d6afc3b477 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:58:25.619112 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:58:25.651362 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 33/67] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jul 13 21:58:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 13 21:58:25.652978 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5f48a2-40a3-4b59-9e72-24597fa981d9 None None] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:58:25.653503 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5f48a2-40a3-4b59-9e72-24597fa981d9 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:58:25.653931 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e5f48a2-40a3-4b59-9e72-24597fa981d9 None None] PATH_INFO: `/` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:58:25.655392 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 34/68] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jul 13 21:58:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 21:58:25.664724 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462c9f71-7ceb-4c33-93b1-238332df90d2 None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:58:25.666288 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462c9f71-7ceb-4c33-93b1-238332df90d2 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:58:25.668353 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03cfa215-11ff-48b3-b32a-930afa8dbd85 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:58:25.668796 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-462c9f71-7ceb-4c33-93b1-238332df90d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:58:25.688241 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-462c9f71-7ceb-4c33-93b1-238332df90d2 None None] MFA Rules not processed for user `cc99c93da5074cf5a48bc2aa9a92c5e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:58:25.689427 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-462c9f71-7ceb-4c33-93b1-238332df90d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:58:25.710989 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb49d75-6299-4b9d-94d8-88005435c580 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:58:25.712187 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb49d75-6299-4b9d-94d8-88005435c580 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:58:25.712628 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb49d75-6299-4b9d-94d8-88005435c580 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:58:25.713055 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb49d75-6299-4b9d-94d8-88005435c580 None admin] PATH_INFO: `/v3/users/4041442c17b840b697a19b97785daf24` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:58:25.728217 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb49d75-6299-4b9d-94d8-88005435c580 None admin] RBAC: Authorizing `identity:get_user(user_id=4041442c17b840b697a19b97785daf24)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:58:25.729810 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb49d75-6299-4b9d-94d8-88005435c580 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:58:25.736193 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 35/69] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Jul 13 21:58:25 2022] GET /identity/v3/users/4041442c17b840b697a19b97785daf24 => generated 275 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 21:58:25.736193 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7232e7-7f9e-4e8b-9e5c-c21be5b021e1 None None] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:58:25.736193 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7232e7-7f9e-4e8b-9e5c-c21be5b021e1 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:58:25.736193 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f7232e7-7f9e-4e8b-9e5c-c21be5b021e1 None None] PATH_INFO: `/` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:58:25.736193 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 36/70] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jul 13 21:58:25 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 21:58:25.736193 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7232e7-7f9e-4e8b-9e5c-c21be5b021e1 None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:58:25.760728 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 35/71] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jul 13 21:58:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 13 21:58:25.761994 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462c9f71-7ceb-4c33-93b1-238332df90d2 None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:58:25.791377 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:58:25.854909 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f7232e7-7f9e-4e8b-9e5c-c21be5b021e1 None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:58:25.870700 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-462c9f71-7ceb-4c33-93b1-238332df90d2 None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:58:25.882191 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:58:25.892420 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7253585-f3ea-43cf-8744-4f78fa352840 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:58:25.893235 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7253585-f3ea-43cf-8744-4f78fa352840 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:58:25.893527 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7253585-f3ea-43cf-8744-4f78fa352840 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:58:25.893807 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7253585-f3ea-43cf-8744-4f78fa352840 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:58:25.904071 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7253585-f3ea-43cf-8744-4f78fa352840 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 21:58:25.907510 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c2ebe1-1914-4cc9-a332-f97959367512 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:58:25.908354 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c2ebe1-1914-4cc9-a332-f97959367512 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:58:25.908799 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c2ebe1-1914-4cc9-a332-f97959367512 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:58:25.909253 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49c2ebe1-1914-4cc9-a332-f97959367512 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:58:25.911713 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application [None req-e7253585-f3ea-43cf-8744-4f78fa352840 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 13 21:58:25.911713 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 21:58:25.911713 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 21:58:25.911713 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 21:58:25.911713 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 21:58:25.911713 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 21:58:25.911713 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 21:58:25.911713 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 21:58:25.911713 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 21:58:25.911713 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 21:58:25.911713 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 21:58:25.911713 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 21:58:25.911713 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 21:58:25.911713 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 21:58:25.911713 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 21:58:25.911713 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 21:58:25.912639 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 21:58:25.912639 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 21:58:25.912639 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 21:58:25.912639 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 21:58:25.912639 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 21:58:25.912639 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 21:58:25.912639 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 21:58:25.912639 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 21:58:25.912639 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 21:58:25.912639 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 21:58:25.912639 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 21:58:25.912639 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application with Lock( Jul 13 21:58:25.912639 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 21:58:25.912639 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._enter() Jul 13 21:58:25.912639 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 21:58:25.912639 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 21:58:25.913430 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 21:58:25.913430 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.creator() Jul 13 21:58:25.913430 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 21:58:25.913430 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application created_value = creator( Jul 13 21:58:25.913430 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 21:58:25.913430 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 21:58:25.913430 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 21:58:25.913430 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 21:58:25.913430 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 21:58:25.913430 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 21:58:25.913430 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 13 21:58:25.913430 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Jul 13 21:58:25.913430 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 37/72] 10.222.0.52 () {56 vars in 1096 bytes} [Wed Jul 13 21:58:25 2022] GET /identity/v3/roles/admin => generated 83 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 21:58:25.914584 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ee42b6-e051-42cf-b944-88d3385e7cc7 None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:58:25.914873 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ee42b6-e051-42cf-b944-88d3385e7cc7 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:58:25.915153 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33ee42b6-e051-42cf-b944-88d3385e7cc7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:58:25.931970 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-33ee42b6-e051-42cf-b944-88d3385e7cc7 None None] MFA Rules not processed for user `cc99c93da5074cf5a48bc2aa9a92c5e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:58:25.933456 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-33ee42b6-e051-42cf-b944-88d3385e7cc7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:58:25.983093 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c2ebe1-1914-4cc9-a332-f97959367512 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:58:25.984598 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49c2ebe1-1914-4cc9-a332-f97959367512 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:58:25.998566 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 38/73] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jul 13 21:58:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 13 21:58:25.999455 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ee42b6-e051-42cf-b944-88d3385e7cc7 None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:58:26.011250 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:58:26.088567 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 36/74] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Jul 13 21:58:25 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 13 21:58:26.089410 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c2ebe1-1914-4cc9-a332-f97959367512 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:58:26.099415 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33ee42b6-e051-42cf-b944-88d3385e7cc7 None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:58:26.118268 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:58:26.136501 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4244e1-bb98-4507-be0c-e00598dbb64f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:58:26.137522 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4244e1-bb98-4507-be0c-e00598dbb64f None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:58:26.137790 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4244e1-bb98-4507-be0c-e00598dbb64f None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:58:26.138066 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4244e1-bb98-4507-be0c-e00598dbb64f None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:58:26.148739 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4244e1-bb98-4507-be0c-e00598dbb64f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 21:58:26.157380 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application [None req-fc4244e1-bb98-4507-be0c-e00598dbb64f None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 13 21:58:26.157380 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 21:58:26.157380 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 21:58:26.157380 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 21:58:26.157380 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 21:58:26.157380 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 21:58:26.157380 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 21:58:26.157380 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 21:58:26.157380 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 21:58:26.157380 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 21:58:26.157380 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 21:58:26.157380 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 21:58:26.157380 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 21:58:26.157380 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 21:58:26.157380 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 21:58:26.157380 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 21:58:26.158230 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 21:58:26.158230 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 21:58:26.158230 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 21:58:26.158230 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 21:58:26.158230 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 21:58:26.158230 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 21:58:26.158230 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 21:58:26.158230 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 21:58:26.158230 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 21:58:26.158230 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 21:58:26.158230 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 21:58:26.158230 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application with Lock( Jul 13 21:58:26.158230 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 21:58:26.158230 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._enter() Jul 13 21:58:26.158230 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 21:58:26.158230 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 21:58:26.159033 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 21:58:26.159033 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.creator() Jul 13 21:58:26.159033 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 21:58:26.159033 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application created_value = creator( Jul 13 21:58:26.159033 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 21:58:26.159033 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 21:58:26.159033 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 21:58:26.159033 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 21:58:26.159033 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 21:58:26.159033 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 21:58:26.159033 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 13 21:58:26.159033 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Jul 13 21:58:26.159033 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 39/75] 10.222.0.52 () {56 vars in 1114 bytes} [Wed Jul 13 21:58:25 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 160 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 21:58:26.160210 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54ee814-4478-41c1-bf47-24833a1ed453 None None] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:58:26.160318 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54ee814-4478-41c1-bf47-24833a1ed453 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:58:26.160559 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a54ee814-4478-41c1-bf47-24833a1ed453 None None] PATH_INFO: `/` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:58:26.161931 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 40/76] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jul 13 21:58:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 21:58:26.162556 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54ee814-4478-41c1-bf47-24833a1ed453 None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:58:26.197658 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49c2ebe1-1914-4cc9-a332-f97959367512 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:58:26.218015 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:58:26.233116 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02434fe-e57f-47a3-8042-6f822e1357a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:58:26.233803 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02434fe-e57f-47a3-8042-6f822e1357a6 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:58:26.234085 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02434fe-e57f-47a3-8042-6f822e1357a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:58:26.234363 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b02434fe-e57f-47a3-8042-6f822e1357a6 None admin] PATH_INFO: `/v3/roles` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:58:26.235164 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02434fe-e57f-47a3-8042-6f822e1357a6 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 21:58:26.235476 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02434fe-e57f-47a3-8042-6f822e1357a6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:58:26.236881 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b02434fe-e57f-47a3-8042-6f822e1357a6 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:58:26.245159 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 37/77] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Jul 13 21:58:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 21:58:26.246518 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02434fe-e57f-47a3-8042-6f822e1357a6 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:58:26.266951 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a54ee814-4478-41c1-bf47-24833a1ed453 None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:58:26.301040 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97415ca-6c9f-49a3-81a6-09c1cb423f77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:58:26.301845 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97415ca-6c9f-49a3-81a6-09c1cb423f77 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:58:26.302127 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97415ca-6c9f-49a3-81a6-09c1cb423f77 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:58:26.302406 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a97415ca-6c9f-49a3-81a6-09c1cb423f77 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:58:26.312507 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a97415ca-6c9f-49a3-81a6-09c1cb423f77 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 21:58:26.320141 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application [None req-a97415ca-6c9f-49a3-81a6-09c1cb423f77 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 13 21:58:26.320141 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 21:58:26.320141 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 21:58:26.320141 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 21:58:26.320141 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 21:58:26.320141 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 21:58:26.320141 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 21:58:26.320141 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 21:58:26.320141 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 21:58:26.320141 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 21:58:26.320141 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 21:58:26.320141 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 21:58:26.320141 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 21:58:26.320141 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 21:58:26.320141 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 21:58:26.320141 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 21:58:26.321182 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 21:58:26.321182 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 21:58:26.321182 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 21:58:26.321182 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 21:58:26.321182 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 21:58:26.321182 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 21:58:26.321182 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 21:58:26.321182 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 21:58:26.321182 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 21:58:26.321182 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 21:58:26.321182 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 21:58:26.321182 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application with Lock( Jul 13 21:58:26.321182 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 21:58:26.321182 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._enter() Jul 13 21:58:26.321182 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 21:58:26.321182 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 21:58:26.322012 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 21:58:26.322012 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.creator() Jul 13 21:58:26.322012 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 21:58:26.322012 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application created_value = creator( Jul 13 21:58:26.322012 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 21:58:26.322012 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 21:58:26.322012 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 21:58:26.322012 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 21:58:26.322012 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 21:58:26.322012 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 21:58:26.322012 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 13 21:58:26.322012 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Jul 13 21:58:26.322012 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 41/78] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Jul 13 21:58:26 2022] GET /identity/v3/roles/member => generated 84 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 21:58:26.322908 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97415ca-6c9f-49a3-81a6-09c1cb423f77 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:58:26.349613 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b02434fe-e57f-47a3-8042-6f822e1357a6 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:58:26.362009 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:58:26.382920 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0c921a5-dcab-4b24-b5f7-0b8828955d0d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:58:26.383800 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c921a5-dcab-4b24-b5f7-0b8828955d0d None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:58:26.384088 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c921a5-dcab-4b24-b5f7-0b8828955d0d None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:58:26.384359 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0c921a5-dcab-4b24-b5f7-0b8828955d0d None admin] PATH_INFO: `/v3/projects/97aa9f72c7a54c9daa7e48288b4aab0f` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:58:26.385918 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c921a5-dcab-4b24-b5f7-0b8828955d0d None admin] RBAC: Authorizing `identity:get_project(project_id=97aa9f72c7a54c9daa7e48288b4aab0f)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:58:26.387990 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0c921a5-dcab-4b24-b5f7-0b8828955d0d None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:58:26.389736 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 38/79] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Jul 13 21:58:26 2022] GET /identity/v3/projects/97aa9f72c7a54c9daa7e48288b4aab0f => generated 298 bytes in 144 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 21:58:26.390912 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be394f4a-ef97-48c0-987d-de6ef0d4d3c6 None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:58:26.391196 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be394f4a-ef97-48c0-987d-de6ef0d4d3c6 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:58:26.391474 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be394f4a-ef97-48c0-987d-de6ef0d4d3c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:58:26.407748 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-be394f4a-ef97-48c0-987d-de6ef0d4d3c6 None None] MFA Rules not processed for user `cc99c93da5074cf5a48bc2aa9a92c5e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:58:26.408930 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-be394f4a-ef97-48c0-987d-de6ef0d4d3c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:58:26.426838 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a97415ca-6c9f-49a3-81a6-09c1cb423f77 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:58:26.453083 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:58:26.460337 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc52d35-e7e6-4d7d-8805-f2074fc3a6da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:58:26.461013 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc52d35-e7e6-4d7d-8805-f2074fc3a6da None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:58:26.461297 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc52d35-e7e6-4d7d-8805-f2074fc3a6da None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:58:26.461576 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc52d35-e7e6-4d7d-8805-f2074fc3a6da None admin] PATH_INFO: `/v3/roles` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:58:26.462335 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc52d35-e7e6-4d7d-8805-f2074fc3a6da None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 21:58:26.462679 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc52d35-e7e6-4d7d-8805-f2074fc3a6da None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:58:26.464162 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc52d35-e7e6-4d7d-8805-f2074fc3a6da None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:58:26.472654 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 42/80] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Jul 13 21:58:26 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 21:58:26.473571 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc52d35-e7e6-4d7d-8805-f2074fc3a6da None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:58:26.474200 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 39/81] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jul 13 21:58:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 13 21:58:26.475019 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be394f4a-ef97-48c0-987d-de6ef0d4d3c6 None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:58:26.576773 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be394f4a-ef97-48c0-987d-de6ef0d4d3c6 None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:58:26.583513 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:58:26.586723 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc52d35-e7e6-4d7d-8805-f2074fc3a6da None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:58:26.611549 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9718445-62ba-4c69-93cd-f12b7a0d3f8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:58:26.612261 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9718445-62ba-4c69-93cd-f12b7a0d3f8a None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:58:26.612450 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9718445-62ba-4c69-93cd-f12b7a0d3f8a None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:58:26.612733 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9718445-62ba-4c69-93cd-f12b7a0d3f8a None admin] PATH_INFO: `/v3/roles` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:58:26.613503 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9718445-62ba-4c69-93cd-f12b7a0d3f8a None admin] RBAC: Adding query filter params (name=member) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 21:58:26.613818 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9718445-62ba-4c69-93cd-f12b7a0d3f8a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:58:26.615242 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9718445-62ba-4c69-93cd-f12b7a0d3f8a None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:58:26.620766 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824741b9-37ab-4a7d-9a33-6235f3f08a61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:58:26.621662 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824741b9-37ab-4a7d-9a33-6235f3f08a61 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:58:26.621946 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824741b9-37ab-4a7d-9a33-6235f3f08a61 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:58:26.622226 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-824741b9-37ab-4a7d-9a33-6235f3f08a61 None admin] PATH_INFO: `/v3/users/cc99c93da5074cf5a48bc2aa9a92c5e7` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:58:26.623299 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 40/82] 10.222.0.52 () {56 vars in 1101 bytes} [Wed Jul 13 21:58:26 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 21:58:26.623485 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824741b9-37ab-4a7d-9a33-6235f3f08a61 None admin] RBAC: Authorizing `identity:get_user(user_id=cc99c93da5074cf5a48bc2aa9a92c5e7)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:58:26.624095 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9718445-62ba-4c69-93cd-f12b7a0d3f8a None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:58:26.624944 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-824741b9-37ab-4a7d-9a33-6235f3f08a61 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:58:26.626627 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 43/83] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Jul 13 21:58:26 2022] GET /identity/v3/users/cc99c93da5074cf5a48bc2aa9a92c5e7 => generated 247 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 21:58:26.631387 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824741b9-37ab-4a7d-9a33-6235f3f08a61 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:58:26.667492 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:58:26.724532 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9718445-62ba-4c69-93cd-f12b7a0d3f8a None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:58:26.734901 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-824741b9-37ab-4a7d-9a33-6235f3f08a61 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:58:26.759439 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23b009c-6e00-431f-9d9d-626dc68856f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:58:26.760091 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23b009c-6e00-431f-9d9d-626dc68856f2 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:58:26.760361 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23b009c-6e00-431f-9d9d-626dc68856f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:58:26.760681 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c23b009c-6e00-431f-9d9d-626dc68856f2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:58:26.761625 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23b009c-6e00-431f-9d9d-626dc68856f2 None admin] RBAC: Adding query filter params (role.id=2a5e86976394498abb6310697cdb3f8b, scope.project.id=97aa9f72c7a54c9daa7e48288b4aab0f, user.id=4041442c17b840b697a19b97785daf24) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 21:58:26.762105 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23b009c-6e00-431f-9d9d-626dc68856f2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:58:26.763585 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c23b009c-6e00-431f-9d9d-626dc68856f2 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:58:26.768914 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 41/84] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Jul 13 21:58:26 2022] GET /identity/v3/role_assignments?user.id=4041442c17b840b697a19b97785daf24&scope.project.id=97aa9f72c7a54c9daa7e48288b4aab0f&role.id=2a5e86976394498abb6310697cdb3f8b => generated 259 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 21:58:26.769673 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23b009c-6e00-431f-9d9d-626dc68856f2 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:58:26.770995 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af5531d-4d7a-4463-aa90-445f28dbcc4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:58:26.771887 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af5531d-4d7a-4463-aa90-445f28dbcc4f None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:58:26.772206 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af5531d-4d7a-4463-aa90-445f28dbcc4f None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:58:26.772445 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6af5531d-4d7a-4463-aa90-445f28dbcc4f None admin] PATH_INFO: `/v3/users/4041442c17b840b697a19b97785daf24` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:58:26.773780 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af5531d-4d7a-4463-aa90-445f28dbcc4f None admin] RBAC: Authorizing `identity:get_user(user_id=4041442c17b840b697a19b97785daf24)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:58:26.775320 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6af5531d-4d7a-4463-aa90-445f28dbcc4f None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:58:26.780728 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 44/85] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Jul 13 21:58:26 2022] GET /identity/v3/users/4041442c17b840b697a19b97785daf24 => generated 275 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 21:58:26.780728 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af5531d-4d7a-4463-aa90-445f28dbcc4f None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:58:26.819574 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:58:26.891322 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6af5531d-4d7a-4463-aa90-445f28dbcc4f None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:58:26.896033 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c23b009c-6e00-431f-9d9d-626dc68856f2 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:58:26.903735 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:58:26.930427 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1d6a1a-f68f-4d6d-b27d-ce380bcff703 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:58:26.931432 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1d6a1a-f68f-4d6d-b27d-ce380bcff703 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:58:26.931736 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1d6a1a-f68f-4d6d-b27d-ce380bcff703 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:58:26.932001 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1d6a1a-f68f-4d6d-b27d-ce380bcff703 None admin] PATH_INFO: `/v3/users/4041442c17b840b697a19b97785daf24` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:58:26.933974 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1d6a1a-f68f-4d6d-b27d-ce380bcff703 None admin] RBAC: Authorizing `identity:get_user(user_id=4041442c17b840b697a19b97785daf24)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:58:26.935493 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1d6a1a-f68f-4d6d-b27d-ce380bcff703 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:58:26.937286 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 45/86] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Jul 13 21:58:26 2022] GET /identity/v3/users/4041442c17b840b697a19b97785daf24 => generated 275 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 21:58:26.938062 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1d6a1a-f68f-4d6d-b27d-ce380bcff703 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:58:26.943554 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71703f6d-6fe3-4854-bc87-c34da85369ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:58:26.944387 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71703f6d-6fe3-4854-bc87-c34da85369ce None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:58:26.944676 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71703f6d-6fe3-4854-bc87-c34da85369ce None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:58:26.944954 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71703f6d-6fe3-4854-bc87-c34da85369ce None admin] PATH_INFO: `/v3/domains/default` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:58:26.946257 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71703f6d-6fe3-4854-bc87-c34da85369ce None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:58:26.947830 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71703f6d-6fe3-4854-bc87-c34da85369ce None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:58:26.950140 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 42/87] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Jul 13 21:58:26 2022] GET /identity/v3/domains/default => generated 199 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 21:58:26.951410 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71703f6d-6fe3-4854-bc87-c34da85369ce None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:58:27.047369 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1d6a1a-f68f-4d6d-b27d-ce380bcff703 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:58:27.050607 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:58:27.054060 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71703f6d-6fe3-4854-bc87-c34da85369ce None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:58:27.085394 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd59dcf-9d2b-4dd2-8942-0c919ecf8b55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:58:27.086055 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd59dcf-9d2b-4dd2-8942-0c919ecf8b55 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:58:27.086342 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd59dcf-9d2b-4dd2-8942-0c919ecf8b55 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:58:27.086653 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cd59dcf-9d2b-4dd2-8942-0c919ecf8b55 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:58:27.089193 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623515c7-ae8f-4f83-9166-17256aad7269 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:58:27.089858 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623515c7-ae8f-4f83-9166-17256aad7269 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:58:27.090145 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623515c7-ae8f-4f83-9166-17256aad7269 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:58:27.090432 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-623515c7-ae8f-4f83-9166-17256aad7269 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:58:27.135095 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:58:27.162842 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623515c7-ae8f-4f83-9166-17256aad7269 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:58:27.163011 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd59dcf-9d2b-4dd2-8942-0c919ecf8b55 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:58:27.164344 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-623515c7-ae8f-4f83-9166-17256aad7269 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:58:27.164503 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cd59dcf-9d2b-4dd2-8942-0c919ecf8b55 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:58:27.269625 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 46/88] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Jul 13 21:58:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 332 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 13 21:58:27.270925 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd59dcf-9d2b-4dd2-8942-0c919ecf8b55 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:58:27.274096 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 43/89] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Jul 13 21:58:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 323 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 13 21:58:27.275062 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623515c7-ae8f-4f83-9166-17256aad7269 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:58:27.276769 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:58:27.351912 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:58:27.378467 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cd59dcf-9d2b-4dd2-8942-0c919ecf8b55 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:58:27.378663 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-623515c7-ae8f-4f83-9166-17256aad7269 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:58:27.413173 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd84232-cdb5-4948-bd83-ab05b5b0d0ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:58:27.414133 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd84232-cdb5-4948-bd83-ab05b5b0d0ea None admin] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:58:27.414622 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd84232-cdb5-4948-bd83-ab05b5b0d0ea None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:58:27.415097 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcd84232-cdb5-4948-bd83-ab05b5b0d0ea None admin] PATH_INFO: `/v3/users` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:58:27.416003 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd84232-cdb5-4948-bd83-ab05b5b0d0ea None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:58:27.417656 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcd84232-cdb5-4948-bd83-ab05b5b0d0ea None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:58:27.418307 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dfce70b-3572-4688-a253-b1eabe1a167d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:58:27.419005 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfce70b-3572-4688-a253-b1eabe1a167d None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:58:27.419294 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfce70b-3572-4688-a253-b1eabe1a167d None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:58:27.419585 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfce70b-3572-4688-a253-b1eabe1a167d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:58:27.444005 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 44/90] 10.222.0.52 () {60 vars in 1132 bytes} [Wed Jul 13 21:58:27 2022] POST /identity/v3/users => generated 289 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 21:58:27.445743 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd84232-cdb5-4948-bd83-ab05b5b0d0ea None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:58:27.522807 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dfce70b-3572-4688-a253-b1eabe1a167d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:58:27.524340 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dfce70b-3572-4688-a253-b1eabe1a167d None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:58:27.528440 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:58:27.594229 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcd84232-cdb5-4948-bd83-ab05b5b0d0ea None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:58:27.635518 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:58:27.643354 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62fb2621-33ad-4a08-9feb-d5b9d3261123 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:58:27.644664 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fb2621-33ad-4a08-9feb-d5b9d3261123 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:58:27.644838 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fb2621-33ad-4a08-9feb-d5b9d3261123 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:58:27.645343 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62fb2621-33ad-4a08-9feb-d5b9d3261123 None admin] PATH_INFO: `/v3/projects/97aa9f72c7a54c9daa7e48288b4aab0f` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:58:27.646879 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62fb2621-33ad-4a08-9feb-d5b9d3261123 None admin] RBAC: Authorizing `identity:get_project(project_id=97aa9f72c7a54c9daa7e48288b4aab0f)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:58:27.648531 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62fb2621-33ad-4a08-9feb-d5b9d3261123 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:58:27.651162 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 45/91] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Jul 13 21:58:27 2022] GET /identity/v3/projects/97aa9f72c7a54c9daa7e48288b4aab0f => generated 298 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 21:58:27.652017 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62fb2621-33ad-4a08-9feb-d5b9d3261123 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:58:27.664275 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 47/92] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Jul 13 21:58:27 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 393 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 13 21:58:27.665408 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dfce70b-3572-4688-a253-b1eabe1a167d None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:58:27.795471 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dfce70b-3572-4688-a253-b1eabe1a167d None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:58:27.795970 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62fb2621-33ad-4a08-9feb-d5b9d3261123 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:58:27.798739 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:58:27.838102 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51eff356-a0b4-4639-8dda-62d30e456c2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:58:27.838859 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51eff356-a0b4-4639-8dda-62d30e456c2e None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:58:27.839129 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51eff356-a0b4-4639-8dda-62d30e456c2e None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:58:27.839395 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51eff356-a0b4-4639-8dda-62d30e456c2e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:58:27.840923 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51eff356-a0b4-4639-8dda-62d30e456c2e None admin] RBAC: Adding query filter params (role.id=db84c6c897534af1a60b9ce62532f383, scope.project.id=97aa9f72c7a54c9daa7e48288b4aab0f, user.id=4041442c17b840b697a19b97785daf24) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 21:58:27.840923 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51eff356-a0b4-4639-8dda-62d30e456c2e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:58:27.841800 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8d7b64-3ac8-406c-b98c-8ae0a293fc4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:58:27.842030 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51eff356-a0b4-4639-8dda-62d30e456c2e None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:58:27.843028 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8d7b64-3ac8-406c-b98c-8ae0a293fc4e None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:58:27.843447 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8d7b64-3ac8-406c-b98c-8ae0a293fc4e None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:58:27.843873 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e8d7b64-3ac8-406c-b98c-8ae0a293fc4e None admin] PATH_INFO: `/v3/projects/97aa9f72c7a54c9daa7e48288b4aab0f` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:58:27.845346 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e8d7b64-3ac8-406c-b98c-8ae0a293fc4e None admin] RBAC: Authorizing `identity:get_project(project_id=97aa9f72c7a54c9daa7e48288b4aab0f)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:58:27.846994 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e8d7b64-3ac8-406c-b98c-8ae0a293fc4e None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:58:27.851206 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 48/94] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Jul 13 21:58:27 2022] GET /identity/v3/role_assignments?user.id=4041442c17b840b697a19b97785daf24&scope.project.id=97aa9f72c7a54c9daa7e48288b4aab0f&role.id=db84c6c897534af1a60b9ce62532f383 => generated 259 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 21:58:27.851206 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51eff356-a0b4-4639-8dda-62d30e456c2e None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:58:27.851536 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 46/94] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Jul 13 21:58:27 2022] GET /identity/v3/projects/97aa9f72c7a54c9daa7e48288b4aab0f => generated 298 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 21:58:27.856138 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8d7b64-3ac8-406c-b98c-8ae0a293fc4e None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:58:27.963319 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:58:28.058935 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51eff356-a0b4-4639-8dda-62d30e456c2e None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:58:28.068529 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e8d7b64-3ac8-406c-b98c-8ae0a293fc4e None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:58:28.147517 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df4aa64-099e-45c3-b7bc-547b8954a7e8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:58:28.148426 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df4aa64-099e-45c3-b7bc-547b8954a7e8 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:58:28.148710 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df4aa64-099e-45c3-b7bc-547b8954a7e8 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:58:28.148978 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5df4aa64-099e-45c3-b7bc-547b8954a7e8 None admin] PATH_INFO: `/v3/projects/3b1b394b75ee4e04aeff0d92911450d5` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:58:28.150289 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df4aa64-099e-45c3-b7bc-547b8954a7e8 None admin] RBAC: Authorizing `identity:get_project(project_id=3b1b394b75ee4e04aeff0d92911450d5)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:58:28.152165 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5df4aa64-099e-45c3-b7bc-547b8954a7e8 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:58:28.154225 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 49/95] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Jul 13 21:58:27 2022] GET /identity/v3/projects/3b1b394b75ee4e04aeff0d92911450d5 => generated 312 bytes in 303 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 21:58:28.155875 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c184be81-5279-4733-9f00-4f78af0e97e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:58:28.156564 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c184be81-5279-4733-9f00-4f78af0e97e4 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:58:28.156850 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c184be81-5279-4733-9f00-4f78af0e97e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:58:28.157129 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c184be81-5279-4733-9f00-4f78af0e97e4 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:58:28.157990 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c184be81-5279-4733-9f00-4f78af0e97e4 None admin] RBAC: Adding query filter params (role.id=44efba2579f342df8a0789c59d163ec3, scope.project.id=97aa9f72c7a54c9daa7e48288b4aab0f, user.id=cc99c93da5074cf5a48bc2aa9a92c5e7) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 21:58:28.158307 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c184be81-5279-4733-9f00-4f78af0e97e4 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:58:28.159888 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df4aa64-099e-45c3-b7bc-547b8954a7e8 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:58:28.163740 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c184be81-5279-4733-9f00-4f78af0e97e4 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:58:28.172869 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 47/96] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Jul 13 21:58:27 2022] GET /identity/v3/role_assignments?user.id=cc99c93da5074cf5a48bc2aa9a92c5e7&scope.project.id=97aa9f72c7a54c9daa7e48288b4aab0f&role.id=44efba2579f342df8a0789c59d163ec3 => generated 259 bytes in 317 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 21:58:28.231914 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:58:28.315425 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5df4aa64-099e-45c3-b7bc-547b8954a7e8 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:58:28.351889 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf2ae364-dbae-469e-9ac1-5fad48c209e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:58:28.352576 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2ae364-dbae-469e-9ac1-5fad48c209e0 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:58:28.352859 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2ae364-dbae-469e-9ac1-5fad48c209e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:58:28.353136 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf2ae364-dbae-469e-9ac1-5fad48c209e0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:58:28.354004 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf2ae364-dbae-469e-9ac1-5fad48c209e0 None admin] RBAC: Adding query filter params (role.id=2a5e86976394498abb6310697cdb3f8b, scope.project.id=3b1b394b75ee4e04aeff0d92911450d5, user.id=4041442c17b840b697a19b97785daf24) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 21:58:28.354317 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf2ae364-dbae-469e-9ac1-5fad48c209e0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:58:28.356062 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf2ae364-dbae-469e-9ac1-5fad48c209e0 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:58:28.362596 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 50/97] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Jul 13 21:58:28 2022] GET /identity/v3/role_assignments?user.id=4041442c17b840b697a19b97785daf24&scope.project.id=3b1b394b75ee4e04aeff0d92911450d5&role.id=2a5e86976394498abb6310697cdb3f8b => generated 259 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 21:58:29.109030 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772d2b16-2740-4ede-ad63-dc58e8869702 None None] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:58:29.110041 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772d2b16-2740-4ede-ad63-dc58e8869702 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:58:29.110569 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-772d2b16-2740-4ede-ad63-dc58e8869702 None None] PATH_INFO: `/` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:58:29.112182 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 48/98] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jul 13 21:58:29 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 21:58:29.117801 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21196964-95ed-4c12-a780-7b2f77969713 None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:58:29.118213 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21196964-95ed-4c12-a780-7b2f77969713 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:58:29.118400 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21196964-95ed-4c12-a780-7b2f77969713 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:58:29.139463 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-21196964-95ed-4c12-a780-7b2f77969713 None None] MFA Rules not processed for user `cc99c93da5074cf5a48bc2aa9a92c5e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:58:29.140656 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-21196964-95ed-4c12-a780-7b2f77969713 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:58:29.218987 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 51/99] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jul 13 21:58:29 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 13 21:58:29.253488 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772d2b16-2740-4ede-ad63-dc58e8869702 None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:58:29.378720 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-772d2b16-2740-4ede-ad63-dc58e8869702 None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:58:29.407538 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:58:29.413721 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa12ffc-7765-4b28-a82a-e204df00e8b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:58:29.414547 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa12ffc-7765-4b28-a82a-e204df00e8b6 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:58:29.414832 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa12ffc-7765-4b28-a82a-e204df00e8b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:58:29.415129 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4aa12ffc-7765-4b28-a82a-e204df00e8b6 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:58:29.426070 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4aa12ffc-7765-4b28-a82a-e204df00e8b6 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 21:58:29.433894 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application [None req-4aa12ffc-7765-4b28-a82a-e204df00e8b6 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 13 21:58:29.433894 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 21:58:29.433894 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 21:58:29.433894 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 21:58:29.433894 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 21:58:29.433894 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 21:58:29.433894 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 21:58:29.433894 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 21:58:29.433894 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 21:58:29.433894 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 21:58:29.433894 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 21:58:29.433894 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 21:58:29.433894 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 21:58:29.433894 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 21:58:29.433894 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 21:58:29.433894 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 21:58:29.434780 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 21:58:29.434780 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 21:58:29.434780 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 21:58:29.434780 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 21:58:29.434780 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 21:58:29.434780 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 21:58:29.434780 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 21:58:29.434780 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 21:58:29.434780 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 21:58:29.434780 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 21:58:29.434780 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 21:58:29.434780 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application with Lock( Jul 13 21:58:29.434780 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 21:58:29.434780 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._enter() Jul 13 21:58:29.434780 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 21:58:29.434780 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 21:58:29.435556 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 21:58:29.435556 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.creator() Jul 13 21:58:29.435556 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 21:58:29.435556 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application created_value = creator( Jul 13 21:58:29.435556 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 21:58:29.435556 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 21:58:29.435556 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 21:58:29.435556 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 21:58:29.435556 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 21:58:29.435556 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 21:58:29.435556 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 13 21:58:29.435556 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Jul 13 21:58:29.435556 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 49/100] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Jul 13 21:58:29 2022] GET /identity/v3/roles/member => generated 84 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 21:58:29.441252 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21196964-95ed-4c12-a780-7b2f77969713 None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:58:29.471932 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:58:29.558137 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21196964-95ed-4c12-a780-7b2f77969713 None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:58:29.608801 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f441d9-f43e-4c64-816b-2909223168fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:58:29.609594 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f441d9-f43e-4c64-816b-2909223168fc None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:58:29.609877 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f441d9-f43e-4c64-816b-2909223168fc None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:58:29.610149 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f441d9-f43e-4c64-816b-2909223168fc None admin] PATH_INFO: `/v3/roles` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:58:29.610943 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f441d9-f43e-4c64-816b-2909223168fc None admin] RBAC: Adding query filter params (name=member) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 21:58:29.611250 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f441d9-f43e-4c64-816b-2909223168fc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:58:29.612710 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f441d9-f43e-4c64-816b-2909223168fc None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:58:29.624339 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 52/101] 10.222.0.52 () {56 vars in 1101 bytes} [Wed Jul 13 21:58:29 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 21:58:29.630424 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa12ffc-7765-4b28-a82a-e204df00e8b6 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:58:29.736108 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4aa12ffc-7765-4b28-a82a-e204df00e8b6 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:58:29.770483 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f2b923-8aee-4b80-b009-7f041f0f6343 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:58:29.771399 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f2b923-8aee-4b80-b009-7f041f0f6343 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:58:29.771679 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f2b923-8aee-4b80-b009-7f041f0f6343 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:58:29.771958 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f2b923-8aee-4b80-b009-7f041f0f6343 None admin] PATH_INFO: `/v3/users/4041442c17b840b697a19b97785daf24` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:58:29.773171 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f2b923-8aee-4b80-b009-7f041f0f6343 None admin] RBAC: Authorizing `identity:get_user(user_id=4041442c17b840b697a19b97785daf24)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:58:29.774765 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f2b923-8aee-4b80-b009-7f041f0f6343 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:58:29.776563 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 50/102] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Jul 13 21:58:29 2022] GET /identity/v3/users/4041442c17b840b697a19b97785daf24 => generated 275 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 21:58:29.782208 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f441d9-f43e-4c64-816b-2909223168fc None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:58:29.896973 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:58:29.907328 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f441d9-f43e-4c64-816b-2909223168fc None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:58:29.941530 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24c51958-597e-4ad9-9860-7e54d473334a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:58:29.942411 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c51958-597e-4ad9-9860-7e54d473334a None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:58:29.942723 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c51958-597e-4ad9-9860-7e54d473334a None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:58:29.942997 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24c51958-597e-4ad9-9860-7e54d473334a None admin] PATH_INFO: `/v3/projects/97aa9f72c7a54c9daa7e48288b4aab0f` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:58:29.944251 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c51958-597e-4ad9-9860-7e54d473334a None admin] RBAC: Authorizing `identity:get_project(project_id=97aa9f72c7a54c9daa7e48288b4aab0f)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:58:29.945768 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24c51958-597e-4ad9-9860-7e54d473334a None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:58:29.947509 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 53/103] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Jul 13 21:58:29 2022] GET /identity/v3/projects/97aa9f72c7a54c9daa7e48288b4aab0f => generated 298 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 21:58:29.953936 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f2b923-8aee-4b80-b009-7f041f0f6343 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:58:30.013179 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:58:30.054762 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f2b923-8aee-4b80-b009-7f041f0f6343 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:58:30.088810 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95e46b78-b6ef-4013-9f3d-5d1dbebef6f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:58:30.089710 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e46b78-b6ef-4013-9f3d-5d1dbebef6f7 None admin] REQUEST_METHOD: `PUT` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:58:30.089987 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e46b78-b6ef-4013-9f3d-5d1dbebef6f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:58:30.090256 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95e46b78-b6ef-4013-9f3d-5d1dbebef6f7 None admin] PATH_INFO: `/v3/projects/97aa9f72c7a54c9daa7e48288b4aab0f/users/4041442c17b840b697a19b97785daf24/roles/2a5e86976394498abb6310697cdb3f8b` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:58:30.092289 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e46b78-b6ef-4013-9f3d-5d1dbebef6f7 None admin] RBAC: Authorizing `identity:create_grant(project_id=97aa9f72c7a54c9daa7e48288b4aab0f, user_id=4041442c17b840b697a19b97785daf24, role_id=2a5e86976394498abb6310697cdb3f8b)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:58:30.093743 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95e46b78-b6ef-4013-9f3d-5d1dbebef6f7 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:58:30.110102 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 51/104] 10.222.0.52 () {58 vars in 1439 bytes} [Wed Jul 13 21:58:29 2022] PUT /identity/v3/projects/97aa9f72c7a54c9daa7e48288b4aab0f/users/4041442c17b840b697a19b97785daf24/roles/2a5e86976394498abb6310697cdb3f8b => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 21:58:30.157988 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3916d5-ecbd-400d-95e5-dc80070ee4c5 None None] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:58:30.158514 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3916d5-ecbd-400d-95e5-dc80070ee4c5 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:58:30.158514 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b3916d5-ecbd-400d-95e5-dc80070ee4c5 None None] PATH_INFO: `/` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:58:30.160205 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 54/105] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jul 13 21:58:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 21:58:30.165673 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f019d10f-1042-4a65-901e-70cae22ccc10 None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:58:30.166045 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f019d10f-1042-4a65-901e-70cae22ccc10 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:58:30.166330 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f019d10f-1042-4a65-901e-70cae22ccc10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:58:30.186967 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-f019d10f-1042-4a65-901e-70cae22ccc10 None None] MFA Rules not processed for user `cc99c93da5074cf5a48bc2aa9a92c5e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:58:30.188421 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-f019d10f-1042-4a65-901e-70cae22ccc10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:58:30.276317 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 52/106] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jul 13 21:58:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 13 21:58:30.311895 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3916d5-ecbd-400d-95e5-dc80070ee4c5 None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:58:30.433546 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b3916d5-ecbd-400d-95e5-dc80070ee4c5 None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:58:30.470570 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ad450c-8909-4912-b48d-4a36a3da420b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:58:30.471470 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ad450c-8909-4912-b48d-4a36a3da420b None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:58:30.471765 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ad450c-8909-4912-b48d-4a36a3da420b None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:58:30.472042 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21ad450c-8909-4912-b48d-4a36a3da420b None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:58:30.489703 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21ad450c-8909-4912-b48d-4a36a3da420b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 21:58:30.498380 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application [None req-21ad450c-8909-4912-b48d-4a36a3da420b None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 13 21:58:30.498380 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 21:58:30.498380 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 21:58:30.498380 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 21:58:30.498380 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 21:58:30.498380 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 21:58:30.498380 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 21:58:30.498380 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 21:58:30.498380 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 21:58:30.498380 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 21:58:30.498380 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 21:58:30.498380 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 21:58:30.498380 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 21:58:30.498380 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 21:58:30.498380 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 21:58:30.498380 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 21:58:30.499279 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 21:58:30.499279 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 21:58:30.499279 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 21:58:30.499279 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 21:58:30.499279 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 21:58:30.499279 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 21:58:30.499279 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 21:58:30.499279 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 21:58:30.499279 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 21:58:30.499279 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 21:58:30.499279 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 21:58:30.499279 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application with Lock( Jul 13 21:58:30.499279 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 21:58:30.499279 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._enter() Jul 13 21:58:30.499279 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 21:58:30.499279 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 21:58:30.500141 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 21:58:30.500141 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.creator() Jul 13 21:58:30.500141 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 21:58:30.500141 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application created_value = creator( Jul 13 21:58:30.500141 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 21:58:30.500141 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 21:58:30.500141 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 21:58:30.500141 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 21:58:30.500141 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 21:58:30.500141 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 21:58:30.500141 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 13 21:58:30.500141 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Jul 13 21:58:30.500141 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 55/107] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Jul 13 21:58:30 2022] GET /identity/v3/roles/reader => generated 84 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 21:58:30.505578 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f019d10f-1042-4a65-901e-70cae22ccc10 None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:58:30.563830 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:58:30.617818 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f019d10f-1042-4a65-901e-70cae22ccc10 None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:58:30.659815 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e971c57-a68f-4b3c-8482-07061b2d7cec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:58:30.660503 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e971c57-a68f-4b3c-8482-07061b2d7cec None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:58:30.660787 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e971c57-a68f-4b3c-8482-07061b2d7cec None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:58:30.661061 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e971c57-a68f-4b3c-8482-07061b2d7cec None admin] PATH_INFO: `/v3/roles` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:58:30.661829 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e971c57-a68f-4b3c-8482-07061b2d7cec None admin] RBAC: Adding query filter params (name=reader) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 21:58:30.662143 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e971c57-a68f-4b3c-8482-07061b2d7cec None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:58:30.668369 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e971c57-a68f-4b3c-8482-07061b2d7cec None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:58:30.677146 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 53/108] 10.222.0.52 () {56 vars in 1101 bytes} [Wed Jul 13 21:58:30 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 21:58:30.683227 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ad450c-8909-4912-b48d-4a36a3da420b None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:58:30.708779 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:58:30.793650 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21ad450c-8909-4912-b48d-4a36a3da420b None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:58:30.835023 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85d6ab06-4489-49ee-bb64-a07e79fa00e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:58:30.835921 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d6ab06-4489-49ee-bb64-a07e79fa00e0 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:58:30.836204 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d6ab06-4489-49ee-bb64-a07e79fa00e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:58:30.836484 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85d6ab06-4489-49ee-bb64-a07e79fa00e0 None admin] PATH_INFO: `/v3/users/3b66c85500f54bc088f91021d5e64666` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:58:30.858581 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d6ab06-4489-49ee-bb64-a07e79fa00e0 None admin] RBAC: Authorizing `identity:get_user(user_id=3b66c85500f54bc088f91021d5e64666)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:58:30.860148 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85d6ab06-4489-49ee-bb64-a07e79fa00e0 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:58:30.861979 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 56/109] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Jul 13 21:58:30 2022] GET /identity/v3/users/3b66c85500f54bc088f91021d5e64666 => generated 289 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 21:58:30.867483 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e971c57-a68f-4b3c-8482-07061b2d7cec None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:58:30.994480 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e971c57-a68f-4b3c-8482-07061b2d7cec None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:58:31.009095 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:58:31.042479 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064ddfd0-919c-4865-ba45-c5e40fc38c53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:58:31.043542 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064ddfd0-919c-4865-ba45-c5e40fc38c53 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:58:31.043826 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064ddfd0-919c-4865-ba45-c5e40fc38c53 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:58:31.044107 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-064ddfd0-919c-4865-ba45-c5e40fc38c53 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:58:31.075148 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b367352c-96e2-4428-b029-b3a0a0a895b2 None None] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:58:31.075452 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b367352c-96e2-4428-b029-b3a0a0a895b2 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:58:31.075708 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b367352c-96e2-4428-b029-b3a0a0a895b2 None None] PATH_INFO: `/` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:58:31.077328 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 57/110] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jul 13 21:58:31 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 13 21:58:31.086110 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ec0bf8-310b-46a4-a90c-551057a01b8a None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:58:31.086392 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ec0bf8-310b-46a4-a90c-551057a01b8a None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:58:31.086699 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42ec0bf8-310b-46a4-a90c-551057a01b8a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:58:31.110946 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-42ec0bf8-310b-46a4-a90c-551057a01b8a None None] MFA Rules not processed for user `cc99c93da5074cf5a48bc2aa9a92c5e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:58:31.112179 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-42ec0bf8-310b-46a4-a90c-551057a01b8a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:58:31.169699 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-064ddfd0-919c-4865-ba45-c5e40fc38c53 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:58:31.171228 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-064ddfd0-919c-4865-ba45-c5e40fc38c53 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:58:31.238566 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 58/111] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jul 13 21:58:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 153 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 13 21:58:31.271705 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ec0bf8-310b-46a4-a90c-551057a01b8a None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:58:31.287921 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:58:31.348457 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 54/112] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Jul 13 21:58:30 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 481 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 13 21:58:31.354674 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064ddfd0-919c-4865-ba45-c5e40fc38c53 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:58:31.374521 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:58:31.444160 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42ec0bf8-310b-46a4-a90c-551057a01b8a None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:58:31.484699 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2812747d-7fe6-4d99-aba7-a667e73b9e89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:58:31.485512 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2812747d-7fe6-4d99-aba7-a667e73b9e89 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:58:31.485794 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2812747d-7fe6-4d99-aba7-a667e73b9e89 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:58:31.486071 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2812747d-7fe6-4d99-aba7-a667e73b9e89 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:58:31.487702 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2812747d-7fe6-4d99-aba7-a667e73b9e89 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:58:31.489318 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2812747d-7fe6-4d99-aba7-a667e73b9e89 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:58:31.491432 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 59/113] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Jul 13 21:58:31 2022] GET /identity/v3/domains/default => generated 199 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 21:58:31.498113 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2812747d-7fe6-4d99-aba7-a667e73b9e89 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:58:31.600479 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-064ddfd0-919c-4865-ba45-c5e40fc38c53 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:58:31.611018 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:58:31.634902 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2812747d-7fe6-4d99-aba7-a667e73b9e89 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:58:31.663221 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c6d8dd2-d061-4940-b1c6-c313dbead055 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:58:31.664121 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6d8dd2-d061-4940-b1c6-c313dbead055 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:58:31.664398 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6d8dd2-d061-4940-b1c6-c313dbead055 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:58:31.664671 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c6d8dd2-d061-4940-b1c6-c313dbead055 None admin] PATH_INFO: `/v3/projects/97aa9f72c7a54c9daa7e48288b4aab0f` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:58:31.665922 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6d8dd2-d061-4940-b1c6-c313dbead055 None admin] RBAC: Authorizing `identity:get_project(project_id=97aa9f72c7a54c9daa7e48288b4aab0f)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:58:31.667747 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c6d8dd2-d061-4940-b1c6-c313dbead055 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:58:31.669518 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 55/114] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Jul 13 21:58:31 2022] GET /identity/v3/projects/97aa9f72c7a54c9daa7e48288b4aab0f => generated 298 bytes in 315 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 21:58:31.670077 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ad8f21f-6e76-49c8-b2b4-35c570cba638 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:58:31.670784 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad8f21f-6e76-49c8-b2b4-35c570cba638 None admin] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:58:31.671073 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad8f21f-6e76-49c8-b2b4-35c570cba638 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:58:31.671351 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ad8f21f-6e76-49c8-b2b4-35c570cba638 None admin] PATH_INFO: `/v3/projects` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:58:31.672068 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad8f21f-6e76-49c8-b2b4-35c570cba638 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:58:31.673512 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ad8f21f-6e76-49c8-b2b4-35c570cba638 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:58:31.675795 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116cbe5d-0c8a-4b17-8ccf-3562df77f6fd None None] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:58:31.676090 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116cbe5d-0c8a-4b17-8ccf-3562df77f6fd None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:58:31.676374 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116cbe5d-0c8a-4b17-8ccf-3562df77f6fd None None] PATH_INFO: `/` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:58:31.677810 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 56/115] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jul 13 21:58:31 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 21:58:31.682347 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116cbe5d-0c8a-4b17-8ccf-3562df77f6fd None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:58:31.711155 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 60/116] 10.222.0.52 () {60 vars in 1141 bytes} [Wed Jul 13 21:58:31 2022] POST /identity/v3/projects => generated 302 bytes in 214 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 13 21:58:31.712597 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c6017f-7a79-4bc5-8791-d4ecc0b5d3ca None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:58:31.712892 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c6017f-7a79-4bc5-8791-d4ecc0b5d3ca None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:58:31.713171 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c6017f-7a79-4bc5-8791-d4ecc0b5d3ca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:58:31.750116 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-88c6017f-7a79-4bc5-8791-d4ecc0b5d3ca None None] MFA Rules not processed for user `cc99c93da5074cf5a48bc2aa9a92c5e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:58:31.755788 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-88c6017f-7a79-4bc5-8791-d4ecc0b5d3ca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:58:31.836123 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 61/117] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jul 13 21:58:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 13 21:58:31.837499 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21b4db8-ff90-4b16-8c3f-aac4961c8ae5 None None] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:58:31.837792 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21b4db8-ff90-4b16-8c3f-aac4961c8ae5 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:58:31.838073 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d21b4db8-ff90-4b16-8c3f-aac4961c8ae5 None None] PATH_INFO: `/` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:58:31.840199 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 62/118] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jul 13 21:58:31 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 21:58:31.844379 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487721c6-1bdd-4100-b4ec-72c46d0056dd None None] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:58:31.845124 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:58:31.846222 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487721c6-1bdd-4100-b4ec-72c46d0056dd None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:58:31.846587 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-487721c6-1bdd-4100-b4ec-72c46d0056dd None None] PATH_INFO: `/` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:58:31.848120 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 63/119] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jul 13 21:58:31 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 13 21:58:31.853848 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c458c6d-7982-497e-851c-2a696efe7731 None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:58:31.854144 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c458c6d-7982-497e-851c-2a696efe7731 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:58:31.854907 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c458c6d-7982-497e-851c-2a696efe7731 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:58:31.874121 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-9c458c6d-7982-497e-851c-2a696efe7731 None None] MFA Rules not processed for user `cc99c93da5074cf5a48bc2aa9a92c5e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:58:31.875564 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-9c458c6d-7982-497e-851c-2a696efe7731 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:58:31.886305 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-116cbe5d-0c8a-4b17-8ccf-3562df77f6fd None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:58:31.926831 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-923be1a8-5e83-46ff-bb12-55911aad11c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:58:31.927516 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923be1a8-5e83-46ff-bb12-55911aad11c1 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:58:31.927799 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923be1a8-5e83-46ff-bb12-55911aad11c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:58:31.928077 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-923be1a8-5e83-46ff-bb12-55911aad11c1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:58:31.929056 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-923be1a8-5e83-46ff-bb12-55911aad11c1 None admin] RBAC: Adding query filter params (role.id=abb8511cb7d54b249e85bbc7507f0317, scope.project.id=97aa9f72c7a54c9daa7e48288b4aab0f, user.id=3b66c85500f54bc088f91021d5e64666) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 21:58:31.929379 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-923be1a8-5e83-46ff-bb12-55911aad11c1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:58:31.930997 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-923be1a8-5e83-46ff-bb12-55911aad11c1 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:58:31.937391 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 57/120] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Jul 13 21:58:31 2022] GET /identity/v3/role_assignments?user.id=3b66c85500f54bc088f91021d5e64666&scope.project.id=97aa9f72c7a54c9daa7e48288b4aab0f&role.id=abb8511cb7d54b249e85bbc7507f0317 => generated 259 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 21:58:31.938751 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4229ef5-946e-466e-9c8c-949a5312f599 None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:58:31.939041 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4229ef5-946e-466e-9c8c-949a5312f599 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:58:31.939327 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4229ef5-946e-466e-9c8c-949a5312f599 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:58:31.947608 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:58:31.958564 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-b4229ef5-946e-466e-9c8c-949a5312f599 None None] MFA Rules not processed for user `cc99c93da5074cf5a48bc2aa9a92c5e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:58:31.960937 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-b4229ef5-946e-466e-9c8c-949a5312f599 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:58:31.967472 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 64/121] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jul 13 21:58:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 13 21:58:31.969028 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c458c6d-7982-497e-851c-2a696efe7731 None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:58:32.039839 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 58/122] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jul 13 21:58:31 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 13 21:58:32.040913 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4229ef5-946e-466e-9c8c-949a5312f599 None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:58:32.109853 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c458c6d-7982-497e-851c-2a696efe7731 None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:58:32.122995 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:58:32.148710 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942be426-c19f-4f80-b0ce-36bb913f2aab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:58:32.149912 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942be426-c19f-4f80-b0ce-36bb913f2aab None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:58:32.150392 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942be426-c19f-4f80-b0ce-36bb913f2aab None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:58:32.150848 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-942be426-c19f-4f80-b0ce-36bb913f2aab None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:58:32.158932 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4229ef5-946e-466e-9c8c-949a5312f599 None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:58:32.165192 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-942be426-c19f-4f80-b0ce-36bb913f2aab None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 21:58:32.175174 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application [None req-942be426-c19f-4f80-b0ce-36bb913f2aab None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 13 21:58:32.175174 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 21:58:32.175174 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 21:58:32.175174 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 21:58:32.175174 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 21:58:32.175174 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 21:58:32.175174 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 21:58:32.175174 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 21:58:32.175174 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 21:58:32.175174 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 21:58:32.175174 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 21:58:32.175174 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 21:58:32.175174 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 21:58:32.175174 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 21:58:32.175174 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 21:58:32.175174 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 21:58:32.176262 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 21:58:32.176262 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 21:58:32.176262 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 21:58:32.176262 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 21:58:32.176262 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 21:58:32.176262 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 21:58:32.176262 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 21:58:32.176262 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 21:58:32.176262 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 21:58:32.176262 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 21:58:32.176262 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 21:58:32.176262 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application with Lock( Jul 13 21:58:32.176262 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 21:58:32.176262 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._enter() Jul 13 21:58:32.176262 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 21:58:32.176262 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 21:58:32.177054 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 21:58:32.177054 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.creator() Jul 13 21:58:32.177054 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 21:58:32.177054 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application created_value = creator( Jul 13 21:58:32.177054 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 21:58:32.177054 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 21:58:32.177054 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 21:58:32.177054 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 21:58:32.177054 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 21:58:32.177054 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 21:58:32.177054 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 13 21:58:32.177054 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Jul 13 21:58:32.181538 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 65/123] 10.222.0.52 () {56 vars in 1114 bytes} [Wed Jul 13 21:58:31 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 21:58:32.181538 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942be426-c19f-4f80-b0ce-36bb913f2aab None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:58:32.207515 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a547e5-9f7e-4226-8cf2-005ee68b4b64 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:58:32.208944 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a547e5-9f7e-4226-8cf2-005ee68b4b64 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:58:32.209393 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a547e5-9f7e-4226-8cf2-005ee68b4b64 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:58:32.209786 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a547e5-9f7e-4226-8cf2-005ee68b4b64 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:58:32.221789 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a547e5-9f7e-4226-8cf2-005ee68b4b64 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 21:58:32.231266 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application [None req-91a547e5-9f7e-4226-8cf2-005ee68b4b64 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 13 21:58:32.231266 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 21:58:32.231266 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 21:58:32.231266 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 21:58:32.231266 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 21:58:32.231266 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 21:58:32.231266 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 21:58:32.231266 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 21:58:32.231266 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 21:58:32.231266 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 21:58:32.231266 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 21:58:32.231266 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 21:58:32.231266 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 21:58:32.231266 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 21:58:32.231266 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 21:58:32.231266 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 21:58:32.232178 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 21:58:32.232178 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 21:58:32.232178 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 21:58:32.232178 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 21:58:32.232178 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 21:58:32.232178 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 21:58:32.232178 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 21:58:32.232178 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 21:58:32.232178 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 21:58:32.232178 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 21:58:32.232178 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 21:58:32.232178 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application with Lock( Jul 13 21:58:32.232178 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 21:58:32.232178 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._enter() Jul 13 21:58:32.232178 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 21:58:32.232178 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 21:58:32.232942 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 21:58:32.232942 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.creator() Jul 13 21:58:32.232942 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 21:58:32.232942 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application created_value = creator( Jul 13 21:58:32.232942 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 21:58:32.232942 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 21:58:32.232942 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 21:58:32.232942 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 21:58:32.232942 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 21:58:32.232942 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 21:58:32.232942 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 13 21:58:32.232942 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Jul 13 21:58:32.235704 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 59/124] 10.222.0.52 () {56 vars in 1096 bytes} [Wed Jul 13 21:58:32 2022] GET /identity/v3/roles/admin => generated 83 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 21:58:32.236877 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a547e5-9f7e-4226-8cf2-005ee68b4b64 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:58:32.284503 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:58:32.392842 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-942be426-c19f-4f80-b0ce-36bb913f2aab None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:58:32.395105 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a547e5-9f7e-4226-8cf2-005ee68b4b64 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:58:32.454571 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:58:32.464251 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d6d277-1620-4b7b-b71a-37029fe2c613 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:58:32.465085 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d6d277-1620-4b7b-b71a-37029fe2c613 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:58:32.465369 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d6d277-1620-4b7b-b71a-37029fe2c613 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:58:32.465648 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2d6d277-1620-4b7b-b71a-37029fe2c613 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:58:32.466772 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261e287a-c212-4f29-8629-acea3459553a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:58:32.467453 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261e287a-c212-4f29-8629-acea3459553a None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:58:32.467789 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261e287a-c212-4f29-8629-acea3459553a None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:58:32.468068 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-261e287a-c212-4f29-8629-acea3459553a None admin] PATH_INFO: `/v3/roles` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:58:32.468915 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261e287a-c212-4f29-8629-acea3459553a None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 21:58:32.469224 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261e287a-c212-4f29-8629-acea3459553a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:58:32.470934 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-261e287a-c212-4f29-8629-acea3459553a None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:58:32.488162 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2d6d277-1620-4b7b-b71a-37029fe2c613 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 21:58:32.491633 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 60/125] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Jul 13 21:58:32 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 255 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 21:58:32.496723 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261e287a-c212-4f29-8629-acea3459553a None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:58:32.506187 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application [None req-b2d6d277-1620-4b7b-b71a-37029fe2c613 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 13 21:58:32.506187 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 21:58:32.506187 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 21:58:32.506187 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 21:58:32.506187 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 21:58:32.506187 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 21:58:32.506187 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 21:58:32.506187 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 21:58:32.506187 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 21:58:32.506187 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 21:58:32.506187 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 21:58:32.506187 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 21:58:32.506187 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 21:58:32.506187 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 21:58:32.506187 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 21:58:32.506187 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 21:58:32.507084 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 21:58:32.507084 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 21:58:32.507084 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 21:58:32.507084 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 21:58:32.507084 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 21:58:32.507084 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 21:58:32.507084 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 21:58:32.507084 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 21:58:32.507084 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 21:58:32.507084 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 21:58:32.507084 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 21:58:32.507084 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application with Lock( Jul 13 21:58:32.507084 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 21:58:32.507084 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._enter() Jul 13 21:58:32.507084 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 21:58:32.507084 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 21:58:32.507840 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 21:58:32.507840 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.creator() Jul 13 21:58:32.507840 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 21:58:32.507840 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application created_value = creator( Jul 13 21:58:32.507840 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 21:58:32.507840 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 21:58:32.507840 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 21:58:32.507840 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 21:58:32.507840 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 21:58:32.507840 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 21:58:32.507840 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 13 21:58:32.507840 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Jul 13 21:58:32.514067 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 66/126] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Jul 13 21:58:32 2022] GET /identity/v3/roles/member => generated 84 bytes in 335 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 21:58:32.515000 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d6d277-1620-4b7b-b71a-37029fe2c613 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:58:32.612018 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-261e287a-c212-4f29-8629-acea3459553a None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:58:32.645774 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36cfecad-9a07-4212-b9f3-f5f6919387a7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:58:32.646853 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cfecad-9a07-4212-b9f3-f5f6919387a7 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:58:32.647341 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cfecad-9a07-4212-b9f3-f5f6919387a7 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:58:32.647800 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36cfecad-9a07-4212-b9f3-f5f6919387a7 None admin] PATH_INFO: `/v3/roles` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:58:32.648739 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36cfecad-9a07-4212-b9f3-f5f6919387a7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 21:58:32.649230 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36cfecad-9a07-4212-b9f3-f5f6919387a7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:58:32.650869 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36cfecad-9a07-4212-b9f3-f5f6919387a7 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:58:32.661353 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 61/127] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Jul 13 21:58:32 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 21:58:32.663661 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36cfecad-9a07-4212-b9f3-f5f6919387a7 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:58:32.727307 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2d6d277-1620-4b7b-b71a-37029fe2c613 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:58:32.746938 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:58:32.754851 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:58:32.794018 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f9056f-0681-4768-bfba-e4d34059551f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:58:32.794944 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f9056f-0681-4768-bfba-e4d34059551f None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:58:32.795231 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f9056f-0681-4768-bfba-e4d34059551f None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:58:32.795506 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16f9056f-0681-4768-bfba-e4d34059551f None admin] PATH_INFO: `/v3/users/4041442c17b840b697a19b97785daf24` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:58:32.796748 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f9056f-0681-4768-bfba-e4d34059551f None admin] RBAC: Authorizing `identity:get_user(user_id=4041442c17b840b697a19b97785daf24)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:58:32.798237 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16f9056f-0681-4768-bfba-e4d34059551f None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:58:32.800015 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 67/128] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Jul 13 21:58:32 2022] GET /identity/v3/users/4041442c17b840b697a19b97785daf24 => generated 275 bytes in 285 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 21:58:32.800578 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36cfecad-9a07-4212-b9f3-f5f6919387a7 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:58:32.811443 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f9056f-0681-4768-bfba-e4d34059551f None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:58:32.841920 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb48e8cf-c43e-47df-83ef-9d1218a85ac6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:58:32.847308 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb48e8cf-c43e-47df-83ef-9d1218a85ac6 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:58:32.847604 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb48e8cf-c43e-47df-83ef-9d1218a85ac6 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:58:32.847894 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb48e8cf-c43e-47df-83ef-9d1218a85ac6 None admin] PATH_INFO: `/v3/roles` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:58:32.848818 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb48e8cf-c43e-47df-83ef-9d1218a85ac6 None admin] RBAC: Adding query filter params (name=member) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 21:58:32.849002 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb48e8cf-c43e-47df-83ef-9d1218a85ac6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:58:32.850438 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb48e8cf-c43e-47df-83ef-9d1218a85ac6 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:58:32.880808 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 62/129] 10.222.0.52 () {56 vars in 1101 bytes} [Wed Jul 13 21:58:32 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 21:58:32.881690 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb48e8cf-c43e-47df-83ef-9d1218a85ac6 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:58:32.940293 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16f9056f-0681-4768-bfba-e4d34059551f None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:58:32.979154 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cafb09c-adf3-41ed-9044-ec2104952227 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:58:32.980541 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cafb09c-adf3-41ed-9044-ec2104952227 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:58:32.981059 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cafb09c-adf3-41ed-9044-ec2104952227 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:58:32.981519 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cafb09c-adf3-41ed-9044-ec2104952227 None admin] PATH_INFO: `/v3/users/cc99c93da5074cf5a48bc2aa9a92c5e7` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:58:32.983150 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cafb09c-adf3-41ed-9044-ec2104952227 None admin] RBAC: Authorizing `identity:get_user(user_id=cc99c93da5074cf5a48bc2aa9a92c5e7)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:58:32.984989 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cafb09c-adf3-41ed-9044-ec2104952227 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:58:32.986862 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 68/130] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Jul 13 21:58:32 2022] GET /identity/v3/users/cc99c93da5074cf5a48bc2aa9a92c5e7 => generated 247 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 21:58:32.988276 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cafb09c-adf3-41ed-9044-ec2104952227 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:58:33.051841 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:58:33.065192 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb48e8cf-c43e-47df-83ef-9d1218a85ac6 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:58:33.084526 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:58:33.102785 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75994b8-d98a-48cf-bc28-e9864641e880 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:58:33.103657 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75994b8-d98a-48cf-bc28-e9864641e880 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:58:33.103939 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75994b8-d98a-48cf-bc28-e9864641e880 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:58:33.104235 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e75994b8-d98a-48cf-bc28-e9864641e880 None admin] PATH_INFO: `/v3/projects/97aa9f72c7a54c9daa7e48288b4aab0f` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:58:33.105578 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75994b8-d98a-48cf-bc28-e9864641e880 None admin] RBAC: Authorizing `identity:get_project(project_id=97aa9f72c7a54c9daa7e48288b4aab0f)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:58:33.107194 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e75994b8-d98a-48cf-bc28-e9864641e880 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:58:33.108921 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 63/131] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Jul 13 21:58:32 2022] GET /identity/v3/projects/97aa9f72c7a54c9daa7e48288b4aab0f => generated 298 bytes in 228 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 21:58:33.114561 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75994b8-d98a-48cf-bc28-e9864641e880 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:58:33.125716 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cafb09c-adf3-41ed-9044-ec2104952227 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:58:33.172285 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec7a32b-476f-461f-92ad-85d86e206c07 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:58:33.174355 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec7a32b-476f-461f-92ad-85d86e206c07 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:58:33.174895 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec7a32b-476f-461f-92ad-85d86e206c07 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:58:33.175590 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec7a32b-476f-461f-92ad-85d86e206c07 None admin] PATH_INFO: `/v3/users/4041442c17b840b697a19b97785daf24` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:58:33.177896 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec7a32b-476f-461f-92ad-85d86e206c07 None admin] RBAC: Authorizing `identity:get_user(user_id=4041442c17b840b697a19b97785daf24)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:58:33.179939 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dec7a32b-476f-461f-92ad-85d86e206c07 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:58:33.183089 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 69/132] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Jul 13 21:58:32 2022] GET /identity/v3/users/4041442c17b840b697a19b97785daf24 => generated 275 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 21:58:33.184677 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190d0e31-28a3-441c-8d03-c94a8441e992 None None] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:58:33.185240 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190d0e31-28a3-441c-8d03-c94a8441e992 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:58:33.185819 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-190d0e31-28a3-441c-8d03-c94a8441e992 None None] PATH_INFO: `/` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:58:33.188253 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 70/133] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jul 13 21:58:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 21:58:33.189126 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190d0e31-28a3-441c-8d03-c94a8441e992 None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:58:33.229672 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e75994b8-d98a-48cf-bc28-e9864641e880 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:58:33.268880 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250087db-60b8-455c-8a30-03b848f30c41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:58:33.269825 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250087db-60b8-455c-8a30-03b848f30c41 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:58:33.270158 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250087db-60b8-455c-8a30-03b848f30c41 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:58:33.270443 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250087db-60b8-455c-8a30-03b848f30c41 None admin] PATH_INFO: `/v3/projects/97aa9f72c7a54c9daa7e48288b4aab0f` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:58:33.271809 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250087db-60b8-455c-8a30-03b848f30c41 None admin] RBAC: Authorizing `identity:get_project(project_id=97aa9f72c7a54c9daa7e48288b4aab0f)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:58:33.273385 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250087db-60b8-455c-8a30-03b848f30c41 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:58:33.275409 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 64/134] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Jul 13 21:58:33 2022] GET /identity/v3/projects/97aa9f72c7a54c9daa7e48288b4aab0f => generated 298 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 21:58:33.276269 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250087db-60b8-455c-8a30-03b848f30c41 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:58:33.305672 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-190d0e31-28a3-441c-8d03-c94a8441e992 None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:58:33.319613 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:58:33.338721 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:58:33.343887 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-50ce4284-2a8b-4108-ade6-abc7c0373788 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:58:33.345099 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ce4284-2a8b-4108-ade6-abc7c0373788 None admin] REQUEST_METHOD: `PUT` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:58:33.345394 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ce4284-2a8b-4108-ade6-abc7c0373788 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:58:33.345912 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-50ce4284-2a8b-4108-ade6-abc7c0373788 None admin] PATH_INFO: `/v3/projects/97aa9f72c7a54c9daa7e48288b4aab0f/users/4041442c17b840b697a19b97785daf24/roles/db84c6c897534af1a60b9ce62532f383` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:58:33.348696 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ce4284-2a8b-4108-ade6-abc7c0373788 None admin] RBAC: Authorizing `identity:create_grant(project_id=97aa9f72c7a54c9daa7e48288b4aab0f, user_id=4041442c17b840b697a19b97785daf24, role_id=db84c6c897534af1a60b9ce62532f383)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:58:33.350292 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-50ce4284-2a8b-4108-ade6-abc7c0373788 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:58:33.371150 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 71/135] 10.222.0.52 () {58 vars in 1439 bytes} [Wed Jul 13 21:58:33 2022] PUT /identity/v3/projects/97aa9f72c7a54c9daa7e48288b4aab0f/users/4041442c17b840b697a19b97785daf24/roles/db84c6c897534af1a60b9ce62532f383 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 21:58:33.373352 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec680e6-ce4f-42f3-82cd-aeff6574dc8c None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:58:33.373352 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec680e6-ce4f-42f3-82cd-aeff6574dc8c None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:58:33.373633 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec680e6-ce4f-42f3-82cd-aeff6574dc8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:58:33.396679 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-1ec680e6-ce4f-42f3-82cd-aeff6574dc8c None None] MFA Rules not processed for user `cc99c93da5074cf5a48bc2aa9a92c5e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:58:33.398374 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-1ec680e6-ce4f-42f3-82cd-aeff6574dc8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:58:33.403825 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250087db-60b8-455c-8a30-03b848f30c41 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:58:33.451777 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df74c28e-63f0-4aa8-95e7-b0781dbaabad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:58:33.453281 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df74c28e-63f0-4aa8-95e7-b0781dbaabad None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:58:33.454206 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df74c28e-63f0-4aa8-95e7-b0781dbaabad None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:58:33.454735 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df74c28e-63f0-4aa8-95e7-b0781dbaabad None admin] PATH_INFO: `/v3/projects/3b1b394b75ee4e04aeff0d92911450d5` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:58:33.456605 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df74c28e-63f0-4aa8-95e7-b0781dbaabad None admin] RBAC: Authorizing `identity:get_project(project_id=3b1b394b75ee4e04aeff0d92911450d5)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:58:33.458687 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df74c28e-63f0-4aa8-95e7-b0781dbaabad None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:58:33.461282 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 65/136] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Jul 13 21:58:33 2022] GET /identity/v3/projects/3b1b394b75ee4e04aeff0d92911450d5 => generated 312 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 21:58:33.462323 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df74c28e-63f0-4aa8-95e7-b0781dbaabad None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:58:33.480800 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 72/137] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jul 13 21:58:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 13 21:58:33.481612 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec680e6-ce4f-42f3-82cd-aeff6574dc8c None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:58:33.622837 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df74c28e-63f0-4aa8-95e7-b0781dbaabad None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:58:33.638016 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec680e6-ce4f-42f3-82cd-aeff6574dc8c None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:58:33.640570 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:58:33.658887 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:58:33.667121 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba25aa8-b729-40ba-8db9-c5c0aa9aca67 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:58:33.668051 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba25aa8-b729-40ba-8db9-c5c0aa9aca67 None admin] REQUEST_METHOD: `PUT` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:58:33.668340 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba25aa8-b729-40ba-8db9-c5c0aa9aca67 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:58:33.668620 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ba25aa8-b729-40ba-8db9-c5c0aa9aca67 None admin] PATH_INFO: `/v3/projects/97aa9f72c7a54c9daa7e48288b4aab0f/users/cc99c93da5074cf5a48bc2aa9a92c5e7/roles/44efba2579f342df8a0789c59d163ec3` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:58:33.670714 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba25aa8-b729-40ba-8db9-c5c0aa9aca67 None admin] RBAC: Authorizing `identity:create_grant(project_id=97aa9f72c7a54c9daa7e48288b4aab0f, user_id=cc99c93da5074cf5a48bc2aa9a92c5e7, role_id=44efba2579f342df8a0789c59d163ec3)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:58:33.672271 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ba25aa8-b729-40ba-8db9-c5c0aa9aca67 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:58:33.684628 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfedddc-8e07-4eea-b5a9-8ae983ec7116 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:58:33.685637 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfedddc-8e07-4eea-b5a9-8ae983ec7116 None admin] REQUEST_METHOD: `PUT` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:58:33.685951 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfedddc-8e07-4eea-b5a9-8ae983ec7116 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:58:33.686286 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4bfedddc-8e07-4eea-b5a9-8ae983ec7116 None admin] PATH_INFO: `/v3/projects/3b1b394b75ee4e04aeff0d92911450d5/users/4041442c17b840b697a19b97785daf24/roles/2a5e86976394498abb6310697cdb3f8b` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:58:33.686825 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 66/138] 10.222.0.52 () {58 vars in 1439 bytes} [Wed Jul 13 21:58:33 2022] PUT /identity/v3/projects/97aa9f72c7a54c9daa7e48288b4aab0f/users/cc99c93da5074cf5a48bc2aa9a92c5e7/roles/44efba2579f342df8a0789c59d163ec3 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 21:58:33.687854 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba25aa8-b729-40ba-8db9-c5c0aa9aca67 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:58:33.688505 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bfedddc-8e07-4eea-b5a9-8ae983ec7116 None admin] RBAC: Authorizing `identity:create_grant(project_id=3b1b394b75ee4e04aeff0d92911450d5, user_id=4041442c17b840b697a19b97785daf24, role_id=2a5e86976394498abb6310697cdb3f8b)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:58:33.691268 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4bfedddc-8e07-4eea-b5a9-8ae983ec7116 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:58:33.712631 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 73/139] 10.222.0.52 () {58 vars in 1439 bytes} [Wed Jul 13 21:58:33 2022] PUT /identity/v3/projects/3b1b394b75ee4e04aeff0d92911450d5/users/4041442c17b840b697a19b97785daf24/roles/2a5e86976394498abb6310697cdb3f8b => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 21:58:33.817636 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ba25aa8-b729-40ba-8db9-c5c0aa9aca67 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:58:33.866532 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2257fbb9-abe3-4578-99bb-ebf9e0c0c8da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:58:33.867354 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2257fbb9-abe3-4578-99bb-ebf9e0c0c8da None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:58:33.867634 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2257fbb9-abe3-4578-99bb-ebf9e0c0c8da None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:58:33.867907 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2257fbb9-abe3-4578-99bb-ebf9e0c0c8da None admin] PATH_INFO: `/v3/roles/member` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:58:33.887426 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2257fbb9-abe3-4578-99bb-ebf9e0c0c8da None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 21:58:33.895510 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application [None req-2257fbb9-abe3-4578-99bb-ebf9e0c0c8da None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 13 21:58:33.895510 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 21:58:33.895510 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 21:58:33.895510 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 21:58:33.895510 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 21:58:33.895510 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 21:58:33.895510 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 21:58:33.895510 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 21:58:33.895510 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 21:58:33.895510 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 21:58:33.895510 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 21:58:33.895510 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 21:58:33.895510 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 21:58:33.895510 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 21:58:33.895510 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 21:58:33.895510 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 21:58:33.896405 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 21:58:33.896405 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 21:58:33.896405 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 21:58:33.896405 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 21:58:33.896405 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 21:58:33.896405 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 21:58:33.896405 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 21:58:33.896405 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 21:58:33.896405 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 21:58:33.896405 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 21:58:33.896405 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 21:58:33.896405 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application with Lock( Jul 13 21:58:33.896405 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 21:58:33.896405 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._enter() Jul 13 21:58:33.896405 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 21:58:33.896405 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 21:58:33.898567 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 21:58:33.898567 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.creator() Jul 13 21:58:33.898567 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 21:58:33.898567 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application created_value = creator( Jul 13 21:58:33.898567 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 21:58:33.898567 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 21:58:33.898567 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 21:58:33.898567 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 21:58:33.898567 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 21:58:33.898567 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 21:58:33.898567 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 13 21:58:33.898567 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Jul 13 21:58:33.898567 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 67/140] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Jul 13 21:58:33 2022] GET /identity/v3/roles/member => generated 84 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 21:58:33.906479 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfedddc-8e07-4eea-b5a9-8ae983ec7116 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:58:34.022378 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4bfedddc-8e07-4eea-b5a9-8ae983ec7116 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:58:34.061095 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1828e5f9-df76-4aaa-a651-2c38a3678714 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:58:34.061797 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1828e5f9-df76-4aaa-a651-2c38a3678714 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:58:34.062071 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1828e5f9-df76-4aaa-a651-2c38a3678714 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:58:34.062342 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1828e5f9-df76-4aaa-a651-2c38a3678714 None admin] PATH_INFO: `/v3/roles` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:58:34.063156 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1828e5f9-df76-4aaa-a651-2c38a3678714 None admin] RBAC: Adding query filter params (name=member) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 21:58:34.063515 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1828e5f9-df76-4aaa-a651-2c38a3678714 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:58:34.064979 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1828e5f9-df76-4aaa-a651-2c38a3678714 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:58:34.075495 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 74/141] 10.222.0.52 () {56 vars in 1101 bytes} [Wed Jul 13 21:58:33 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 21:58:34.081016 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2257fbb9-abe3-4578-99bb-ebf9e0c0c8da None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:58:34.129161 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:58:34.193496 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2257fbb9-abe3-4578-99bb-ebf9e0c0c8da None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:58:34.229652 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef572d7-b97f-4723-b0bf-41cf50e238d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:58:34.230579 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef572d7-b97f-4723-b0bf-41cf50e238d2 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:58:34.230863 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef572d7-b97f-4723-b0bf-41cf50e238d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:58:34.231136 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef572d7-b97f-4723-b0bf-41cf50e238d2 None admin] PATH_INFO: `/v3/users/4041442c17b840b697a19b97785daf24` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:58:34.232365 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef572d7-b97f-4723-b0bf-41cf50e238d2 None admin] RBAC: Authorizing `identity:get_user(user_id=4041442c17b840b697a19b97785daf24)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:58:34.233910 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef572d7-b97f-4723-b0bf-41cf50e238d2 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:58:34.235888 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 68/142] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Jul 13 21:58:34 2022] GET /identity/v3/users/4041442c17b840b697a19b97785daf24 => generated 275 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 21:58:34.241197 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1828e5f9-df76-4aaa-a651-2c38a3678714 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:58:34.269319 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:58:34.353368 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4207ee6f-3241-4dce-9260-f5e078f29280 None None] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:58:34.353546 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4207ee6f-3241-4dce-9260-f5e078f29280 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:58:34.353826 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4207ee6f-3241-4dce-9260-f5e078f29280 None None] PATH_INFO: `/` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:58:34.355446 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 69/143] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jul 13 21:58:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 13 21:58:34.363079 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ababb2ec-03ee-440d-a0b7-d742333d2899 None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:58:34.363384 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ababb2ec-03ee-440d-a0b7-d742333d2899 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:58:34.363634 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ababb2ec-03ee-440d-a0b7-d742333d2899 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:58:34.378289 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1828e5f9-df76-4aaa-a651-2c38a3678714 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:58:34.393905 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-ababb2ec-03ee-440d-a0b7-d742333d2899 None None] MFA Rules not processed for user `cc99c93da5074cf5a48bc2aa9a92c5e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:58:34.395144 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-ababb2ec-03ee-440d-a0b7-d742333d2899 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:58:34.424640 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d0f263-81b4-4a39-8f76-eca46c7836ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:58:34.425907 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d0f263-81b4-4a39-8f76-eca46c7836ad None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:58:34.426197 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d0f263-81b4-4a39-8f76-eca46c7836ad None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:58:34.426473 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00d0f263-81b4-4a39-8f76-eca46c7836ad None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:58:34.469791 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 70/144] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jul 13 21:58:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 13 21:58:34.471346 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b502c0-272a-4c6a-86b4-027d43b191c1 None None] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:58:34.471751 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b502c0-272a-4c6a-86b4-027d43b191c1 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:58:34.472122 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b502c0-272a-4c6a-86b4-027d43b191c1 None None] PATH_INFO: `/` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:58:34.473639 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 71/145] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jul 13 21:58:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 21:58:34.479668 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec60d6a2-cea2-4e64-b1f6-de054f6d1aa9 None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:58:34.480477 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec60d6a2-cea2-4e64-b1f6-de054f6d1aa9 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:58:34.480859 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec60d6a2-cea2-4e64-b1f6-de054f6d1aa9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:58:34.506941 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-ec60d6a2-cea2-4e64-b1f6-de054f6d1aa9 None None] MFA Rules not processed for user `cc99c93da5074cf5a48bc2aa9a92c5e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:58:34.508989 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-ec60d6a2-cea2-4e64-b1f6-de054f6d1aa9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:58:34.525833 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d0f263-81b4-4a39-8f76-eca46c7836ad None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:58:34.527410 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00d0f263-81b4-4a39-8f76-eca46c7836ad None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:58:34.578375 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:58:34.583967 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:58:34.644713 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 72/146] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jul 13 21:58:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 13 21:58:34.646181 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec60d6a2-cea2-4e64-b1f6-de054f6d1aa9 None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:58:34.717816 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 75/147] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Jul 13 21:58:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 477 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 13 21:58:34.719000 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d0f263-81b4-4a39-8f76-eca46c7836ad None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:58:34.821509 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec60d6a2-cea2-4e64-b1f6-de054f6d1aa9 None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:58:34.825243 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00d0f263-81b4-4a39-8f76-eca46c7836ad None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:58:34.851026 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:58:34.856224 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b386efc0-aa53-4b5a-9796-15b02c78e752 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:58:34.857045 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b386efc0-aa53-4b5a-9796-15b02c78e752 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:58:34.857321 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b386efc0-aa53-4b5a-9796-15b02c78e752 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:58:34.857589 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b386efc0-aa53-4b5a-9796-15b02c78e752 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:58:34.862942 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec606b2-9558-4eb8-9e6a-c148fee02238 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:58:34.863951 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec606b2-9558-4eb8-9e6a-c148fee02238 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:58:34.864237 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec606b2-9558-4eb8-9e6a-c148fee02238 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:58:34.864608 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec606b2-9558-4eb8-9e6a-c148fee02238 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:58:34.866069 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec606b2-9558-4eb8-9e6a-c148fee02238 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:58:34.867808 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec606b2-9558-4eb8-9e6a-c148fee02238 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:58:34.868136 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b386efc0-aa53-4b5a-9796-15b02c78e752 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 21:58:34.869622 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 76/148] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Jul 13 21:58:34 2022] GET /identity/v3/domains/default => generated 199 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 21:58:34.874767 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec606b2-9558-4eb8-9e6a-c148fee02238 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:58:34.875974 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application [None req-b386efc0-aa53-4b5a-9796-15b02c78e752 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 13 21:58:34.875974 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 21:58:34.875974 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 21:58:34.875974 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 21:58:34.875974 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 21:58:34.875974 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 21:58:34.875974 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 21:58:34.875974 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 21:58:34.875974 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 21:58:34.875974 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 21:58:34.875974 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 21:58:34.875974 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 21:58:34.875974 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 21:58:34.875974 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 21:58:34.875974 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 21:58:34.875974 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 21:58:34.876819 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 21:58:34.876819 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 21:58:34.876819 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 21:58:34.876819 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 21:58:34.876819 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 21:58:34.876819 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 21:58:34.876819 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 21:58:34.876819 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 21:58:34.876819 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 21:58:34.876819 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 21:58:34.876819 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 21:58:34.876819 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application with Lock( Jul 13 21:58:34.876819 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 21:58:34.876819 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._enter() Jul 13 21:58:34.876819 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 21:58:34.876819 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 21:58:34.877593 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 21:58:34.877593 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.creator() Jul 13 21:58:34.877593 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 21:58:34.877593 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application created_value = creator( Jul 13 21:58:34.877593 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 21:58:34.877593 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 21:58:34.877593 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 21:58:34.877593 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 21:58:34.877593 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 21:58:34.877593 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 21:58:34.877593 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 13 21:58:34.877593 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Jul 13 21:58:34.877593 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 73/149] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Jul 13 21:58:34 2022] GET /identity/v3/roles/reader => generated 84 bytes in 232 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 21:58:34.882866 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b386efc0-aa53-4b5a-9796-15b02c78e752 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:58:34.914253 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:58:34.982624 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec606b2-9558-4eb8-9e6a-c148fee02238 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:58:35.019458 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52dd42c-55dd-496c-be51-78f33f03d3a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:58:35.020346 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52dd42c-55dd-496c-be51-78f33f03d3a0 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:58:35.020622 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52dd42c-55dd-496c-be51-78f33f03d3a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:58:35.020894 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e52dd42c-55dd-496c-be51-78f33f03d3a0 None admin] PATH_INFO: `/v3/projects/97aa9f72c7a54c9daa7e48288b4aab0f` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:58:35.022099 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52dd42c-55dd-496c-be51-78f33f03d3a0 None admin] RBAC: Authorizing `identity:get_project(project_id=97aa9f72c7a54c9daa7e48288b4aab0f)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:58:35.023589 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e52dd42c-55dd-496c-be51-78f33f03d3a0 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:58:35.025491 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 77/150] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Jul 13 21:58:34 2022] GET /identity/v3/projects/97aa9f72c7a54c9daa7e48288b4aab0f => generated 298 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 21:58:35.026211 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52dd42c-55dd-496c-be51-78f33f03d3a0 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:58:35.086593 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:58:35.128418 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e52dd42c-55dd-496c-be51-78f33f03d3a0 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:58:35.148687 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b386efc0-aa53-4b5a-9796-15b02c78e752 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:58:35.162271 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e37d53-4bab-4b82-82cf-3ee1b4864e7c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:58:35.162987 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e37d53-4bab-4b82-82cf-3ee1b4864e7c None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:58:35.163275 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e37d53-4bab-4b82-82cf-3ee1b4864e7c None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:58:35.163556 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e37d53-4bab-4b82-82cf-3ee1b4864e7c None admin] PATH_INFO: `/v3/roles` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:58:35.164378 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e37d53-4bab-4b82-82cf-3ee1b4864e7c None admin] RBAC: Adding query filter params (name=reader) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 21:58:35.164631 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e37d53-4bab-4b82-82cf-3ee1b4864e7c None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:58:35.166115 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5e37d53-4bab-4b82-82cf-3ee1b4864e7c None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:58:35.175306 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 78/151] 10.222.0.52 () {56 vars in 1101 bytes} [Wed Jul 13 21:58:35 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 21:58:35.176029 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e37d53-4bab-4b82-82cf-3ee1b4864e7c None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:58:35.235954 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b8d4e7-856e-4915-bb0d-7666f4e7e65b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:58:35.236643 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b8d4e7-856e-4915-bb0d-7666f4e7e65b None admin] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:58:35.236914 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b8d4e7-856e-4915-bb0d-7666f4e7e65b None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:58:35.237190 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b8d4e7-856e-4915-bb0d-7666f4e7e65b None admin] PATH_INFO: `/v3/users` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:58:35.237917 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b8d4e7-856e-4915-bb0d-7666f4e7e65b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:58:35.239427 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87b8d4e7-856e-4915-bb0d-7666f4e7e65b None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:58:35.272762 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 74/152] 10.222.0.52 () {60 vars in 1132 bytes} [Wed Jul 13 21:58:34 2022] POST /identity/v3/users => generated 283 bytes in 390 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 21:58:35.274128 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b8d4e7-856e-4915-bb0d-7666f4e7e65b None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:58:35.315880 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e37d53-4bab-4b82-82cf-3ee1b4864e7c None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:58:35.369852 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc80c67-6cd7-4d12-9fa9-3337476b3dee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:58:35.370681 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc80c67-6cd7-4d12-9fa9-3337476b3dee None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:58:35.371030 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc80c67-6cd7-4d12-9fa9-3337476b3dee None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:58:35.371382 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fc80c67-6cd7-4d12-9fa9-3337476b3dee None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:58:35.372314 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc80c67-6cd7-4d12-9fa9-3337476b3dee None admin] RBAC: Adding query filter params (role.id=2a5e86976394498abb6310697cdb3f8b, scope.project.id=97aa9f72c7a54c9daa7e48288b4aab0f, user.id=4041442c17b840b697a19b97785daf24) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 21:58:35.372701 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc80c67-6cd7-4d12-9fa9-3337476b3dee None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:58:35.374292 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5fc80c67-6cd7-4d12-9fa9-3337476b3dee None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:58:35.378887 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:58:35.387810 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 79/153] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Jul 13 21:58:35 2022] GET /identity/v3/role_assignments?user.id=4041442c17b840b697a19b97785daf24&scope.project.id=97aa9f72c7a54c9daa7e48288b4aab0f&role.id=2a5e86976394498abb6310697cdb3f8b => generated 608 bytes in 212 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 21:58:35.480347 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:58:35.501992 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b8d4e7-856e-4915-bb0d-7666f4e7e65b None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:58:35.550530 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cae1adb-cc24-46f6-84a4-7448275d2400 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:58:35.550530 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cae1adb-cc24-46f6-84a4-7448275d2400 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:58:35.550530 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cae1adb-cc24-46f6-84a4-7448275d2400 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:58:35.550530 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cae1adb-cc24-46f6-84a4-7448275d2400 None admin] PATH_INFO: `/v3/users/3b66c85500f54bc088f91021d5e64666` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:58:35.550946 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cae1adb-cc24-46f6-84a4-7448275d2400 None admin] RBAC: Authorizing `identity:get_user(user_id=3b66c85500f54bc088f91021d5e64666)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:58:35.551437 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cae1adb-cc24-46f6-84a4-7448275d2400 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:58:35.553309 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 75/154] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Jul 13 21:58:35 2022] GET /identity/v3/users/3b66c85500f54bc088f91021d5e64666 => generated 289 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 21:58:35.559046 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc80c67-6cd7-4d12-9fa9-3337476b3dee None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:58:35.718412 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fc80c67-6cd7-4d12-9fa9-3337476b3dee None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:58:35.758984 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ec98ed6-9333-468c-aca4-ed193210e721 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:58:35.759797 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec98ed6-9333-468c-aca4-ed193210e721 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:58:35.760074 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec98ed6-9333-468c-aca4-ed193210e721 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:58:35.760346 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ec98ed6-9333-468c-aca4-ed193210e721 None admin] PATH_INFO: `/v3/projects/97aa9f72c7a54c9daa7e48288b4aab0f` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:58:35.761731 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec98ed6-9333-468c-aca4-ed193210e721 None admin] RBAC: Authorizing `identity:get_project(project_id=97aa9f72c7a54c9daa7e48288b4aab0f)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:58:35.763275 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ec98ed6-9333-468c-aca4-ed193210e721 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:58:35.765365 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 80/155] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Jul 13 21:58:35 2022] GET /identity/v3/projects/97aa9f72c7a54c9daa7e48288b4aab0f => generated 298 bytes in 207 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 21:58:35.771180 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cae1adb-cc24-46f6-84a4-7448275d2400 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:58:35.895854 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cae1adb-cc24-46f6-84a4-7448275d2400 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:58:35.934198 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c3210a-5ae2-449f-96ae-ee9cd556780d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:58:35.935144 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c3210a-5ae2-449f-96ae-ee9cd556780d None admin] REQUEST_METHOD: `PUT` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:58:35.935425 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c3210a-5ae2-449f-96ae-ee9cd556780d None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:58:35.935705 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c3210a-5ae2-449f-96ae-ee9cd556780d None admin] PATH_INFO: `/v3/projects/97aa9f72c7a54c9daa7e48288b4aab0f/users/3b66c85500f54bc088f91021d5e64666/roles/abb8511cb7d54b249e85bbc7507f0317` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:58:35.937741 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c3210a-5ae2-449f-96ae-ee9cd556780d None admin] RBAC: Authorizing `identity:create_grant(project_id=97aa9f72c7a54c9daa7e48288b4aab0f, user_id=3b66c85500f54bc088f91021d5e64666, role_id=abb8511cb7d54b249e85bbc7507f0317)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:58:35.939275 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c3210a-5ae2-449f-96ae-ee9cd556780d None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:58:35.956641 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 76/156] 10.222.0.52 () {58 vars in 1439 bytes} [Wed Jul 13 21:58:35 2022] PUT /identity/v3/projects/97aa9f72c7a54c9daa7e48288b4aab0f/users/3b66c85500f54bc088f91021d5e64666/roles/abb8511cb7d54b249e85bbc7507f0317 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 21:58:36.091741 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7b1e43-58cb-4624-9619-9090f81df046 None None] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:58:36.092042 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7b1e43-58cb-4624-9619-9090f81df046 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:58:36.092211 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e7b1e43-58cb-4624-9619-9090f81df046 None None] PATH_INFO: `/` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:58:36.093823 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 81/157] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jul 13 21:58:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 21:58:36.101789 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3539ef8d-163a-4672-b804-035e3adbc970 None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:58:36.102432 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3539ef8d-163a-4672-b804-035e3adbc970 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:58:36.103122 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3539ef8d-163a-4672-b804-035e3adbc970 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:58:36.123487 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-3539ef8d-163a-4672-b804-035e3adbc970 None None] MFA Rules not processed for user `cc99c93da5074cf5a48bc2aa9a92c5e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:58:36.125042 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-3539ef8d-163a-4672-b804-035e3adbc970 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:58:36.140760 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:58:36.207550 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 77/158] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jul 13 21:58:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 13 21:58:36.231543 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7b1e43-58cb-4624-9619-9090f81df046 None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:58:36.300692 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:58:36.364256 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e7b1e43-58cb-4624-9619-9090f81df046 None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:58:36.408183 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e439071-99bd-4063-9e3a-27759b5e5796 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:58:36.409013 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e439071-99bd-4063-9e3a-27759b5e5796 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:58:36.409296 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e439071-99bd-4063-9e3a-27759b5e5796 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:58:36.409564 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e439071-99bd-4063-9e3a-27759b5e5796 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:58:36.431439 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e439071-99bd-4063-9e3a-27759b5e5796 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 21:58:36.439733 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application [None req-7e439071-99bd-4063-9e3a-27759b5e5796 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 13 21:58:36.439733 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 21:58:36.439733 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 21:58:36.439733 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 21:58:36.439733 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 21:58:36.439733 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 21:58:36.439733 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 21:58:36.439733 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 21:58:36.439733 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 21:58:36.439733 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 21:58:36.439733 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 21:58:36.439733 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 21:58:36.439733 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 21:58:36.439733 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 21:58:36.439733 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 21:58:36.439733 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 21:58:36.440729 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 21:58:36.440729 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 21:58:36.440729 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 21:58:36.440729 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 21:58:36.440729 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 21:58:36.440729 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 21:58:36.440729 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 21:58:36.440729 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 21:58:36.440729 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 21:58:36.440729 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 21:58:36.440729 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 21:58:36.440729 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application with Lock( Jul 13 21:58:36.440729 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 21:58:36.440729 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._enter() Jul 13 21:58:36.440729 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 21:58:36.440729 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 21:58:36.441513 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 21:58:36.441513 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.creator() Jul 13 21:58:36.441513 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 21:58:36.441513 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application created_value = creator( Jul 13 21:58:36.441513 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 21:58:36.441513 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 21:58:36.441513 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 21:58:36.441513 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 21:58:36.441513 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 21:58:36.441513 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 21:58:36.441513 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 13 21:58:36.441513 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Jul 13 21:58:36.441513 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 82/159] 10.222.0.52 () {56 vars in 1114 bytes} [Wed Jul 13 21:58:36 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 21:58:36.451723 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3539ef8d-163a-4672-b804-035e3adbc970 None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:58:36.602160 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3539ef8d-163a-4672-b804-035e3adbc970 None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:58:36.639490 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4645d3-35c2-49b5-994d-0cb11c547431 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:58:36.640561 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4645d3-35c2-49b5-994d-0cb11c547431 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:58:36.641046 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4645d3-35c2-49b5-994d-0cb11c547431 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:58:36.641522 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4645d3-35c2-49b5-994d-0cb11c547431 None admin] PATH_INFO: `/v3/roles` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:58:36.642513 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f4645d3-35c2-49b5-994d-0cb11c547431 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 21:58:36.643017 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f4645d3-35c2-49b5-994d-0cb11c547431 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:58:36.644639 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f4645d3-35c2-49b5-994d-0cb11c547431 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:58:36.655748 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 78/160] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Jul 13 21:58:36 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 21:58:36.661739 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e439071-99bd-4063-9e3a-27759b5e5796 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:58:36.774271 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e439071-99bd-4063-9e3a-27759b5e5796 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:58:36.789779 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:58:36.814009 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f9b0bff-da01-4d42-a7eb-c0257ee7fc2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:58:36.815305 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9b0bff-da01-4d42-a7eb-c0257ee7fc2e None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:58:36.815786 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9b0bff-da01-4d42-a7eb-c0257ee7fc2e None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:58:36.816316 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f9b0bff-da01-4d42-a7eb-c0257ee7fc2e None admin] PATH_INFO: `/v3/users/4041442c17b840b697a19b97785daf24` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:58:36.817752 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9b0bff-da01-4d42-a7eb-c0257ee7fc2e None admin] RBAC: Authorizing `identity:get_user(user_id=4041442c17b840b697a19b97785daf24)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:58:36.819453 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f9b0bff-da01-4d42-a7eb-c0257ee7fc2e None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:58:36.821312 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 83/161] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Jul 13 21:58:36 2022] GET /identity/v3/users/4041442c17b840b697a19b97785daf24 => generated 275 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 21:58:36.827561 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4645d3-35c2-49b5-994d-0cb11c547431 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:58:36.838875 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6cd977-2508-46c7-a86c-5a65e5d53333 None None] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:58:36.839515 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:58:36.843088 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6cd977-2508-46c7-a86c-5a65e5d53333 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:58:36.846762 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b6cd977-2508-46c7-a86c-5a65e5d53333 None None] PATH_INFO: `/` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:58:36.846762 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 84/162] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jul 13 21:58:36 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 21:58:36.852859 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13797220-ab8a-45e8-b902-89f1f0272a48 None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:58:36.853387 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13797220-ab8a-45e8-b902-89f1f0272a48 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:58:36.853856 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13797220-ab8a-45e8-b902-89f1f0272a48 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:58:36.885727 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-13797220-ab8a-45e8-b902-89f1f0272a48 None None] MFA Rules not processed for user `cc99c93da5074cf5a48bc2aa9a92c5e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:58:36.887336 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-13797220-ab8a-45e8-b902-89f1f0272a48 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:58:37.028525 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4645d3-35c2-49b5-994d-0cb11c547431 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:58:37.031237 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 85/163] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jul 13 21:58:36 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 13 21:58:37.032628 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3016ddf1-54ce-496e-a9b9-b47a995e236a None None] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:58:37.032880 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3016ddf1-54ce-496e-a9b9-b47a995e236a None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:58:37.033161 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3016ddf1-54ce-496e-a9b9-b47a995e236a None None] PATH_INFO: `/` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:58:37.034594 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 86/164] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jul 13 21:58:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 21:58:37.040775 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7174eefd-e160-458d-8c8e-eb9c0280fc73 None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:58:37.041063 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7174eefd-e160-458d-8c8e-eb9c0280fc73 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:58:37.041339 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7174eefd-e160-458d-8c8e-eb9c0280fc73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:58:37.072248 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-7174eefd-e160-458d-8c8e-eb9c0280fc73 None None] MFA Rules not processed for user `cc99c93da5074cf5a48bc2aa9a92c5e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:58:37.073586 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-7174eefd-e160-458d-8c8e-eb9c0280fc73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:58:37.089344 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c13b8c-e325-4eb2-9708-b3494bae159e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:58:37.090020 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c13b8c-e325-4eb2-9708-b3494bae159e None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:58:37.090305 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c13b8c-e325-4eb2-9708-b3494bae159e None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:58:37.090654 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21c13b8c-e325-4eb2-9708-b3494bae159e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:58:37.274794 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 87/165] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jul 13 21:58:37 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 235 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 13 21:58:37.276567 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7174eefd-e160-458d-8c8e-eb9c0280fc73 None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:58:37.303576 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c13b8c-e325-4eb2-9708-b3494bae159e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:58:37.305014 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21c13b8c-e325-4eb2-9708-b3494bae159e None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:58:37.317321 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:58:37.406219 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:58:37.621500 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 79/166] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Jul 13 21:58:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 795 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 13 21:58:37.623034 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c13b8c-e325-4eb2-9708-b3494bae159e None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:58:37.652495 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7174eefd-e160-458d-8c8e-eb9c0280fc73 None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:58:37.759774 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36f45f4-3d85-4d10-bfeb-e55027158603 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:58:37.760957 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36f45f4-3d85-4d10-bfeb-e55027158603 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:58:37.761378 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36f45f4-3d85-4d10-bfeb-e55027158603 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:58:37.761792 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f36f45f4-3d85-4d10-bfeb-e55027158603 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:58:37.777722 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f36f45f4-3d85-4d10-bfeb-e55027158603 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 21:58:37.794165 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application [None req-f36f45f4-3d85-4d10-bfeb-e55027158603 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 13 21:58:37.794165 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 21:58:37.794165 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 21:58:37.794165 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 21:58:37.794165 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 21:58:37.794165 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 21:58:37.794165 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 21:58:37.794165 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 21:58:37.794165 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 21:58:37.794165 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 21:58:37.794165 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 21:58:37.794165 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 21:58:37.794165 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 21:58:37.794165 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 21:58:37.794165 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 21:58:37.794165 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 21:58:37.795331 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 21:58:37.795331 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 21:58:37.795331 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 21:58:37.795331 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 21:58:37.795331 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 21:58:37.795331 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 21:58:37.795331 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 21:58:37.795331 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 21:58:37.795331 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 21:58:37.795331 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 21:58:37.795331 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 21:58:37.795331 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application with Lock( Jul 13 21:58:37.795331 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 21:58:37.795331 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._enter() Jul 13 21:58:37.795331 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 21:58:37.795331 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 21:58:37.796526 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 21:58:37.796526 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.creator() Jul 13 21:58:37.796526 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 21:58:37.796526 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application created_value = creator( Jul 13 21:58:37.796526 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 21:58:37.796526 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 21:58:37.796526 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 21:58:37.796526 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 21:58:37.796526 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 21:58:37.796526 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 21:58:37.796526 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 13 21:58:37.796526 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Jul 13 21:58:37.802862 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 88/167] 10.222.0.52 () {56 vars in 1096 bytes} [Wed Jul 13 21:58:37 2022] GET /identity/v3/roles/admin => generated 83 bytes in 527 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 21:58:37.803904 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36f45f4-3d85-4d10-bfeb-e55027158603 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:58:37.879726 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21c13b8c-e325-4eb2-9708-b3494bae159e None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:58:37.895926 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:58:37.959197 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3213fc-7fc9-4c69-b62b-d83a05341c05 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:58:37.960266 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3213fc-7fc9-4c69-b62b-d83a05341c05 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:58:37.960697 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3213fc-7fc9-4c69-b62b-d83a05341c05 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:58:37.961188 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df3213fc-7fc9-4c69-b62b-d83a05341c05 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:58:37.980020 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df3213fc-7fc9-4c69-b62b-d83a05341c05 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 21:58:37.992136 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application [None req-df3213fc-7fc9-4c69-b62b-d83a05341c05 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 13 21:58:37.992136 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 21:58:37.992136 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 21:58:37.992136 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 21:58:37.992136 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 21:58:37.992136 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 21:58:37.992136 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 21:58:37.992136 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 21:58:37.992136 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 21:58:37.992136 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 21:58:37.992136 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 21:58:37.992136 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 21:58:37.992136 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 21:58:37.992136 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 21:58:37.992136 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 21:58:37.992136 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 21:58:37.993058 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 21:58:37.993058 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 21:58:37.993058 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 21:58:37.993058 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 21:58:37.993058 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 21:58:37.993058 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 21:58:37.993058 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 21:58:37.993058 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 21:58:37.993058 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 21:58:37.993058 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 21:58:37.993058 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 21:58:37.993058 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application with Lock( Jul 13 21:58:37.993058 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 21:58:37.993058 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._enter() Jul 13 21:58:37.993058 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 21:58:37.993058 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 21:58:37.993853 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 21:58:37.993853 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.creator() Jul 13 21:58:37.993853 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 21:58:37.993853 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application created_value = creator( Jul 13 21:58:37.993853 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 21:58:37.993853 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 21:58:37.993853 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 21:58:37.993853 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 21:58:37.993853 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 21:58:37.993853 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 21:58:37.993853 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 13 21:58:37.993853 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Jul 13 21:58:37.996340 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 80/168] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Jul 13 21:58:37 2022] GET /identity/v3/roles/member => generated 84 bytes in 374 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 21:58:37.997308 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3213fc-7fc9-4c69-b62b-d83a05341c05 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:58:38.058937 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f36f45f4-3d85-4d10-bfeb-e55027158603 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:58:38.060549 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:58:38.135283 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ad8191-ff28-409c-a8ad-c15691958de6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:58:38.136121 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ad8191-ff28-409c-a8ad-c15691958de6 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:58:38.136403 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ad8191-ff28-409c-a8ad-c15691958de6 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:58:38.136861 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9ad8191-ff28-409c-a8ad-c15691958de6 None admin] PATH_INFO: `/v3/projects/97aa9f72c7a54c9daa7e48288b4aab0f` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:58:38.143213 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ad8191-ff28-409c-a8ad-c15691958de6 None admin] RBAC: Authorizing `identity:get_project(project_id=97aa9f72c7a54c9daa7e48288b4aab0f)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:58:38.144852 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9ad8191-ff28-409c-a8ad-c15691958de6 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:58:38.147152 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 89/169] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Jul 13 21:58:37 2022] GET /identity/v3/projects/97aa9f72c7a54c9daa7e48288b4aab0f => generated 298 bytes in 343 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 21:58:38.148143 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ad8191-ff28-409c-a8ad-c15691958de6 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:58:38.242733 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df3213fc-7fc9-4c69-b62b-d83a05341c05 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:58:38.311311 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83b30d2-f4d2-45e5-bdc8-b76e91c07cd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:58:38.312243 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83b30d2-f4d2-45e5-bdc8-b76e91c07cd0 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:58:38.312683 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83b30d2-f4d2-45e5-bdc8-b76e91c07cd0 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:58:38.313191 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f83b30d2-f4d2-45e5-bdc8-b76e91c07cd0 None admin] PATH_INFO: `/v3/roles` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:58:38.314096 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83b30d2-f4d2-45e5-bdc8-b76e91c07cd0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 21:58:38.314558 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83b30d2-f4d2-45e5-bdc8-b76e91c07cd0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:58:38.316081 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f83b30d2-f4d2-45e5-bdc8-b76e91c07cd0 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:58:38.329903 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 81/170] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Jul 13 21:58:37 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 333 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 21:58:38.331033 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83b30d2-f4d2-45e5-bdc8-b76e91c07cd0 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:58:38.376126 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9ad8191-ff28-409c-a8ad-c15691958de6 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:58:38.387704 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:58:38.467872 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ce8be6-252b-48b4-9bda-f50b7529b6b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:58:38.468569 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ce8be6-252b-48b4-9bda-f50b7529b6b7 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:58:38.468852 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ce8be6-252b-48b4-9bda-f50b7529b6b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:58:38.469135 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1ce8be6-252b-48b4-9bda-f50b7529b6b7 None admin] PATH_INFO: `/v3/roles` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:58:38.469904 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ce8be6-252b-48b4-9bda-f50b7529b6b7 None admin] RBAC: Adding query filter params (name=member) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 21:58:38.470213 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ce8be6-252b-48b4-9bda-f50b7529b6b7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:58:38.471651 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f1ce8be6-252b-48b4-9bda-f50b7529b6b7 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:58:38.492796 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 90/171] 10.222.0.52 () {56 vars in 1101 bytes} [Wed Jul 13 21:58:38 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 345 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 21:58:38.493761 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ce8be6-252b-48b4-9bda-f50b7529b6b7 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:58:38.562865 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:58:38.568009 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f83b30d2-f4d2-45e5-bdc8-b76e91c07cd0 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:58:38.651296 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8425ef8a-ccaa-49e0-b2f3-46f11f2ff650 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:58:38.652065 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8425ef8a-ccaa-49e0-b2f3-46f11f2ff650 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:58:38.652309 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8425ef8a-ccaa-49e0-b2f3-46f11f2ff650 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:58:38.652581 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8425ef8a-ccaa-49e0-b2f3-46f11f2ff650 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:58:38.653439 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8425ef8a-ccaa-49e0-b2f3-46f11f2ff650 None admin] RBAC: Adding query filter params (role.id=db84c6c897534af1a60b9ce62532f383, scope.project.id=97aa9f72c7a54c9daa7e48288b4aab0f, user.id=4041442c17b840b697a19b97785daf24) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 21:58:38.653745 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8425ef8a-ccaa-49e0-b2f3-46f11f2ff650 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:58:38.655224 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8425ef8a-ccaa-49e0-b2f3-46f11f2ff650 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:58:38.664721 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 82/172] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Jul 13 21:58:38 2022] GET /identity/v3/role_assignments?user.id=4041442c17b840b697a19b97785daf24&scope.project.id=97aa9f72c7a54c9daa7e48288b4aab0f&role.id=db84c6c897534af1a60b9ce62532f383 => generated 608 bytes in 334 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 21:58:38.665613 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8425ef8a-ccaa-49e0-b2f3-46f11f2ff650 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:58:38.710426 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1ce8be6-252b-48b4-9bda-f50b7529b6b7 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:58:38.773900 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421ca59a-0865-4b65-ba93-74712b789159 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:58:38.774831 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421ca59a-0865-4b65-ba93-74712b789159 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:58:38.775107 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421ca59a-0865-4b65-ba93-74712b789159 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:58:38.775378 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-421ca59a-0865-4b65-ba93-74712b789159 None admin] PATH_INFO: `/v3/users/cc99c93da5074cf5a48bc2aa9a92c5e7` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:58:38.776640 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421ca59a-0865-4b65-ba93-74712b789159 None admin] RBAC: Authorizing `identity:get_user(user_id=cc99c93da5074cf5a48bc2aa9a92c5e7)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:58:38.778214 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-421ca59a-0865-4b65-ba93-74712b789159 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:58:38.779899 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 91/173] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Jul 13 21:58:38 2022] GET /identity/v3/users/cc99c93da5074cf5a48bc2aa9a92c5e7 => generated 247 bytes in 287 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 21:58:38.787470 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421ca59a-0865-4b65-ba93-74712b789159 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:58:38.915652 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:58:38.923078 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-421ca59a-0865-4b65-ba93-74712b789159 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:58:38.926810 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:58:38.945527 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8425ef8a-ccaa-49e0-b2f3-46f11f2ff650 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:58:38.960708 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-722fdef6-230c-41df-8c45-b0c7ff6b028d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:58:38.961378 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722fdef6-230c-41df-8c45-b0c7ff6b028d None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:58:38.961663 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722fdef6-230c-41df-8c45-b0c7ff6b028d None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:58:38.961928 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-722fdef6-230c-41df-8c45-b0c7ff6b028d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:58:39.032693 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa7ece0f-f891-450e-96ef-130d9d02f027 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:58:39.033601 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7ece0f-f891-450e-96ef-130d9d02f027 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:58:39.033881 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7ece0f-f891-450e-96ef-130d9d02f027 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:58:39.034154 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa7ece0f-f891-450e-96ef-130d9d02f027 None admin] PATH_INFO: `/v3/users/4041442c17b840b697a19b97785daf24` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:58:39.036344 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa7ece0f-f891-450e-96ef-130d9d02f027 None admin] RBAC: Authorizing `identity:get_user(user_id=4041442c17b840b697a19b97785daf24)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:58:39.037821 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa7ece0f-f891-450e-96ef-130d9d02f027 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:58:39.042943 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 83/174] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Jul 13 21:58:38 2022] GET /identity/v3/users/4041442c17b840b697a19b97785daf24 => generated 275 bytes in 378 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 21:58:39.047401 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396585b6-b860-4d17-ad68-c60c27462f70 None None] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:58:39.048841 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396585b6-b860-4d17-ad68-c60c27462f70 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:58:39.049423 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396585b6-b860-4d17-ad68-c60c27462f70 None None] PATH_INFO: `/` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:58:39.052212 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 84/175] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jul 13 21:58:39 2022] GET /identity => generated 270 bytes in 9 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 21:58:39.053912 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396585b6-b860-4d17-ad68-c60c27462f70 None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:58:39.086222 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722fdef6-230c-41df-8c45-b0c7ff6b028d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:58:39.088062 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-722fdef6-230c-41df-8c45-b0c7ff6b028d None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:58:39.188243 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396585b6-b860-4d17-ad68-c60c27462f70 None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:58:39.205745 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:58:39.214848 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 92/176] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Jul 13 21:58:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 428 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 13 21:58:39.217198 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905a7e6e-f3aa-4581-8149-5139ba515c3a None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:58:39.217710 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905a7e6e-f3aa-4581-8149-5139ba515c3a None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:58:39.218185 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-905a7e6e-f3aa-4581-8149-5139ba515c3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:58:39.239239 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a4afaa0-821f-4ef7-9e14-16405860f51f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:58:39.239964 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4afaa0-821f-4ef7-9e14-16405860f51f None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:58:39.240249 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4afaa0-821f-4ef7-9e14-16405860f51f None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:58:39.240526 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a4afaa0-821f-4ef7-9e14-16405860f51f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:58:39.240957 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-905a7e6e-f3aa-4581-8149-5139ba515c3a None None] MFA Rules not processed for user `cc99c93da5074cf5a48bc2aa9a92c5e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:58:39.242355 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-905a7e6e-f3aa-4581-8149-5139ba515c3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:58:39.284565 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:58:39.331939 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 93/177] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jul 13 21:58:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 13 21:58:39.333356 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542d2443-5167-40c1-afc2-fd35f3ae8357 None None] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:58:39.333650 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542d2443-5167-40c1-afc2-fd35f3ae8357 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:58:39.333928 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-542d2443-5167-40c1-afc2-fd35f3ae8357 None None] PATH_INFO: `/` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:58:39.335575 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 94/178] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jul 13 21:58:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 21:58:39.336284 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542d2443-5167-40c1-afc2-fd35f3ae8357 None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:58:39.337887 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4afaa0-821f-4ef7-9e14-16405860f51f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:58:39.343274 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a4afaa0-821f-4ef7-9e14-16405860f51f None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:58:39.461107 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-542d2443-5167-40c1-afc2-fd35f3ae8357 None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:58:39.463693 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 85/179] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Jul 13 21:58:39 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 410 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 13 21:58:39.464953 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64063be4-387f-4d86-b5d1-33fd5c089a72 None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:58:39.465253 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64063be4-387f-4d86-b5d1-33fd5c089a72 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:58:39.466524 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64063be4-387f-4d86-b5d1-33fd5c089a72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:58:39.481505 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-64063be4-387f-4d86-b5d1-33fd5c089a72 None None] MFA Rules not processed for user `cc99c93da5074cf5a48bc2aa9a92c5e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:58:39.482726 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-64063be4-387f-4d86-b5d1-33fd5c089a72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:58:39.493621 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:58:39.500951 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea07bc8-0f3c-46aa-9393-6eade9ffbd01 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:58:39.501829 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea07bc8-0f3c-46aa-9393-6eade9ffbd01 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:58:39.502108 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea07bc8-0f3c-46aa-9393-6eade9ffbd01 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:58:39.502379 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ea07bc8-0f3c-46aa-9393-6eade9ffbd01 None admin] PATH_INFO: `/v3/projects/97aa9f72c7a54c9daa7e48288b4aab0f` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:58:39.503614 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea07bc8-0f3c-46aa-9393-6eade9ffbd01 None admin] RBAC: Authorizing `identity:get_project(project_id=97aa9f72c7a54c9daa7e48288b4aab0f)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:58:39.505070 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ea07bc8-0f3c-46aa-9393-6eade9ffbd01 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:58:39.507296 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 95/180] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Jul 13 21:58:39 2022] GET /identity/v3/projects/97aa9f72c7a54c9daa7e48288b4aab0f => generated 298 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 21:58:39.507589 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea07bc8-0f3c-46aa-9393-6eade9ffbd01 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:58:39.547287 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:58:39.566517 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 86/181] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jul 13 21:58:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 13 21:58:39.566517 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09271591-d07f-48fa-9382-90a175ad96ba None None] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:58:39.566517 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09271591-d07f-48fa-9382-90a175ad96ba None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:58:39.566517 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09271591-d07f-48fa-9382-90a175ad96ba None None] PATH_INFO: `/` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:58:39.566517 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 87/182] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jul 13 21:58:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 21:58:39.566517 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09271591-d07f-48fa-9382-90a175ad96ba None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:58:39.628114 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ea07bc8-0f3c-46aa-9393-6eade9ffbd01 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:58:39.663942 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f33975-0a04-49c2-8685-6a3d64a174f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:58:39.664862 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f33975-0a04-49c2-8685-6a3d64a174f0 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:58:39.665203 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f33975-0a04-49c2-8685-6a3d64a174f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:58:39.665544 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f33975-0a04-49c2-8685-6a3d64a174f0 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:58:39.675990 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51f33975-0a04-49c2-8685-6a3d64a174f0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 21:58:39.682639 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09271591-d07f-48fa-9382-90a175ad96ba None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:58:39.683730 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application [None req-51f33975-0a04-49c2-8685-6a3d64a174f0 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 13 21:58:39.683730 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 21:58:39.683730 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 21:58:39.683730 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 21:58:39.683730 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 21:58:39.683730 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 21:58:39.683730 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 21:58:39.683730 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 21:58:39.683730 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 21:58:39.683730 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 21:58:39.683730 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 21:58:39.683730 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 21:58:39.683730 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 21:58:39.683730 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 21:58:39.683730 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 21:58:39.683730 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 21:58:39.684578 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 21:58:39.684578 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 21:58:39.684578 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 21:58:39.684578 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 21:58:39.684578 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 21:58:39.684578 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 21:58:39.684578 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 21:58:39.684578 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 21:58:39.684578 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 21:58:39.684578 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 21:58:39.684578 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 21:58:39.684578 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application with Lock( Jul 13 21:58:39.684578 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 21:58:39.684578 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._enter() Jul 13 21:58:39.684578 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 21:58:39.684578 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 21:58:39.685406 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 21:58:39.685406 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.creator() Jul 13 21:58:39.685406 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 21:58:39.685406 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application created_value = creator( Jul 13 21:58:39.685406 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 21:58:39.685406 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 21:58:39.685406 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 21:58:39.685406 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 21:58:39.685406 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 21:58:39.685406 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 21:58:39.685406 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 13 21:58:39.685406 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Jul 13 21:58:39.685406 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 96/183] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Jul 13 21:58:39 2022] GET /identity/v3/roles/reader => generated 84 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 21:58:39.686079 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f33975-0a04-49c2-8685-6a3d64a174f0 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:58:39.724875 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc0c0af-35e7-4b5e-b489-c70b4dbf0603 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:58:39.725672 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc0c0af-35e7-4b5e-b489-c70b4dbf0603 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:58:39.725949 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc0c0af-35e7-4b5e-b489-c70b4dbf0603 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:58:39.726222 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc0c0af-35e7-4b5e-b489-c70b4dbf0603 None admin] PATH_INFO: `/v3/projects/3b1b394b75ee4e04aeff0d92911450d5` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:58:39.727664 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc0c0af-35e7-4b5e-b489-c70b4dbf0603 None admin] RBAC: Authorizing `identity:get_project(project_id=3b1b394b75ee4e04aeff0d92911450d5)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:58:39.729656 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc0c0af-35e7-4b5e-b489-c70b4dbf0603 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:58:39.736125 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 88/184] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Jul 13 21:58:39 2022] GET /identity/v3/projects/3b1b394b75ee4e04aeff0d92911450d5 => generated 312 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 21:58:39.736125 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb108fa-75be-4f42-9cd1-e668582fdafe None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:58:39.736125 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb108fa-75be-4f42-9cd1-e668582fdafe None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:58:39.736125 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb108fa-75be-4f42-9cd1-e668582fdafe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:58:39.749755 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-deb108fa-75be-4f42-9cd1-e668582fdafe None None] MFA Rules not processed for user `cc99c93da5074cf5a48bc2aa9a92c5e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:58:39.750077 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:58:39.751629 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-deb108fa-75be-4f42-9cd1-e668582fdafe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:58:39.791850 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f33975-0a04-49c2-8685-6a3d64a174f0 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:58:39.803154 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:58:39.816714 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 89/185] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jul 13 21:58:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 13 21:58:39.817774 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb108fa-75be-4f42-9cd1-e668582fdafe None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:58:39.825471 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc47a6a6-6505-4243-8843-aa66978f9cb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:58:39.826139 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc47a6a6-6505-4243-8843-aa66978f9cb7 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:58:39.826423 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc47a6a6-6505-4243-8843-aa66978f9cb7 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:58:39.826732 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc47a6a6-6505-4243-8843-aa66978f9cb7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:58:39.827623 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc47a6a6-6505-4243-8843-aa66978f9cb7 None admin] RBAC: Adding query filter params (role.id=44efba2579f342df8a0789c59d163ec3, scope.project.id=97aa9f72c7a54c9daa7e48288b4aab0f, user.id=cc99c93da5074cf5a48bc2aa9a92c5e7) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 21:58:39.827932 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc47a6a6-6505-4243-8843-aa66978f9cb7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:58:39.829390 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc47a6a6-6505-4243-8843-aa66978f9cb7 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:58:39.835141 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 97/186] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Jul 13 21:58:39 2022] GET /identity/v3/role_assignments?user.id=cc99c93da5074cf5a48bc2aa9a92c5e7&scope.project.id=97aa9f72c7a54c9daa7e48288b4aab0f&role.id=44efba2579f342df8a0789c59d163ec3 => generated 608 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 21:58:39.835915 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc47a6a6-6505-4243-8843-aa66978f9cb7 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:58:39.938938 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb108fa-75be-4f42-9cd1-e668582fdafe None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:58:39.961920 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc47a6a6-6505-4243-8843-aa66978f9cb7 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:58:39.976335 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ee02da-b721-4c6a-9f7e-e438fdb0c57c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:58:39.977060 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ee02da-b721-4c6a-9f7e-e438fdb0c57c None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:58:39.977336 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ee02da-b721-4c6a-9f7e-e438fdb0c57c None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:58:39.977594 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ee02da-b721-4c6a-9f7e-e438fdb0c57c None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:58:39.987929 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:58:39.989327 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ee02da-b721-4c6a-9f7e-e438fdb0c57c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 21:58:39.997196 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application [None req-76ee02da-b721-4c6a-9f7e-e438fdb0c57c None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 13 21:58:39.997196 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 21:58:39.997196 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 21:58:39.997196 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 21:58:39.997196 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 21:58:39.997196 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 21:58:39.997196 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 21:58:39.997196 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 21:58:39.997196 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 21:58:39.997196 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 21:58:39.997196 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 21:58:39.997196 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 21:58:39.997196 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 21:58:39.997196 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 21:58:39.997196 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 21:58:39.997196 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 21:58:39.998097 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 21:58:39.998097 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 21:58:39.998097 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 21:58:39.998097 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 21:58:39.998097 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 21:58:39.998097 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 21:58:39.998097 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 21:58:39.998097 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 21:58:39.998097 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 21:58:39.998097 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 21:58:39.998097 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 21:58:39.998097 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application with Lock( Jul 13 21:58:39.998097 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 21:58:39.998097 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._enter() Jul 13 21:58:39.998097 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 21:58:39.998097 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 21:58:39.998943 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 21:58:39.998943 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.creator() Jul 13 21:58:39.998943 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 21:58:39.998943 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application created_value = creator( Jul 13 21:58:39.998943 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 21:58:39.998943 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 21:58:39.998943 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 21:58:39.998943 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 21:58:39.998943 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 21:58:39.998943 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 21:58:39.998943 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 13 21:58:39.998943 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Jul 13 21:58:39.998943 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 90/187] 10.222.0.52 () {56 vars in 1096 bytes} [Wed Jul 13 21:58:39 2022] GET /identity/v3/roles/admin => generated 83 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 21:58:39.999593 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ee02da-b721-4c6a-9f7e-e438fdb0c57c None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:58:40.010233 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-273cd24e-5be9-4793-9b15-019e67644da2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:58:40.011127 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273cd24e-5be9-4793-9b15-019e67644da2 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:58:40.011430 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273cd24e-5be9-4793-9b15-019e67644da2 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:58:40.011705 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-273cd24e-5be9-4793-9b15-019e67644da2 None admin] PATH_INFO: `/v3/roles` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:58:40.012557 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273cd24e-5be9-4793-9b15-019e67644da2 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 21:58:40.012811 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273cd24e-5be9-4793-9b15-019e67644da2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:58:40.014204 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-273cd24e-5be9-4793-9b15-019e67644da2 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:58:40.023643 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 98/188] 10.222.0.52 () {56 vars in 1101 bytes} [Wed Jul 13 21:58:39 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 21:58:40.024930 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35962e2-c284-43b4-ab97-36c7c2b74b3d None None] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:58:40.025215 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35962e2-c284-43b4-ab97-36c7c2b74b3d None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:58:40.025546 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e35962e2-c284-43b4-ab97-36c7c2b74b3d None None] PATH_INFO: `/` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:58:40.026940 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 99/189] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jul 13 21:58:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 21:58:40.027601 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35962e2-c284-43b4-ab97-36c7c2b74b3d None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:58:40.058232 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:58:40.109583 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ee02da-b721-4c6a-9f7e-e438fdb0c57c None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:58:40.137581 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e35962e2-c284-43b4-ab97-36c7c2b74b3d None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:58:40.141943 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a04d5c-f40a-4a48-bcfd-9b69ed9e4944 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:58:40.142634 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a04d5c-f40a-4a48-bcfd-9b69ed9e4944 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:58:40.142915 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a04d5c-f40a-4a48-bcfd-9b69ed9e4944 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:58:40.143196 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15a04d5c-f40a-4a48-bcfd-9b69ed9e4944 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:58:40.144046 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a04d5c-f40a-4a48-bcfd-9b69ed9e4944 None admin] RBAC: Adding query filter params (role.id=2a5e86976394498abb6310697cdb3f8b, scope.project.id=3b1b394b75ee4e04aeff0d92911450d5, user.id=4041442c17b840b697a19b97785daf24) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 21:58:40.144352 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a04d5c-f40a-4a48-bcfd-9b69ed9e4944 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:58:40.145818 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15a04d5c-f40a-4a48-bcfd-9b69ed9e4944 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:58:40.151645 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 91/190] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Jul 13 21:58:39 2022] GET /identity/v3/role_assignments?user.id=4041442c17b840b697a19b97785daf24&scope.project.id=3b1b394b75ee4e04aeff0d92911450d5&role.id=2a5e86976394498abb6310697cdb3f8b => generated 608 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 21:58:40.152422 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a04d5c-f40a-4a48-bcfd-9b69ed9e4944 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:58:40.173183 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c5b4b6e-0d3b-477e-9fc5-87b9905de088 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:58:40.174030 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5b4b6e-0d3b-477e-9fc5-87b9905de088 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:58:40.174305 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5b4b6e-0d3b-477e-9fc5-87b9905de088 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:58:40.174605 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c5b4b6e-0d3b-477e-9fc5-87b9905de088 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:58:40.186771 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c5b4b6e-0d3b-477e-9fc5-87b9905de088 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 21:58:40.195094 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application [None req-9c5b4b6e-0d3b-477e-9fc5-87b9905de088 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 13 21:58:40.195094 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 21:58:40.195094 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 21:58:40.195094 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 21:58:40.195094 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 21:58:40.195094 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 21:58:40.195094 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 21:58:40.195094 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 21:58:40.195094 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 21:58:40.195094 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 21:58:40.195094 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 21:58:40.195094 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 21:58:40.195094 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 21:58:40.195094 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 21:58:40.195094 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 21:58:40.195094 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 21:58:40.196060 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 21:58:40.196060 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 21:58:40.196060 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 21:58:40.196060 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 21:58:40.196060 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 21:58:40.196060 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 21:58:40.196060 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 21:58:40.196060 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 21:58:40.196060 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 21:58:40.196060 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 21:58:40.196060 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 21:58:40.196060 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application with Lock( Jul 13 21:58:40.196060 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 21:58:40.196060 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._enter() Jul 13 21:58:40.196060 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 21:58:40.196060 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 21:58:40.196863 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 21:58:40.196863 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.creator() Jul 13 21:58:40.196863 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 21:58:40.196863 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application created_value = creator( Jul 13 21:58:40.196863 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 21:58:40.196863 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 21:58:40.196863 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 21:58:40.196863 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 21:58:40.196863 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 21:58:40.196863 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 21:58:40.196863 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 13 21:58:40.196863 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Jul 13 21:58:40.196863 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 100/191] 10.222.0.52 () {56 vars in 1114 bytes} [Wed Jul 13 21:58:40 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 21:58:40.198258 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135d9060-8a0d-4aa6-ad6f-d0f4e3b18ca8 None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:58:40.198258 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135d9060-8a0d-4aa6-ad6f-d0f4e3b18ca8 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:58:40.198549 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-135d9060-8a0d-4aa6-ad6f-d0f4e3b18ca8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:58:40.219353 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-135d9060-8a0d-4aa6-ad6f-d0f4e3b18ca8 None None] MFA Rules not processed for user `cc99c93da5074cf5a48bc2aa9a92c5e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:58:40.220733 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-135d9060-8a0d-4aa6-ad6f-d0f4e3b18ca8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:58:40.268174 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:58:40.286948 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15a04d5c-f40a-4a48-bcfd-9b69ed9e4944 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:58:40.297268 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 101/192] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jul 13 21:58:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 13 21:58:40.298788 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135d9060-8a0d-4aa6-ad6f-d0f4e3b18ca8 None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:58:40.305512 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:58:40.340658 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8003eba-c8e6-4d54-acae-ac6c436694f2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:58:40.341346 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8003eba-c8e6-4d54-acae-ac6c436694f2 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:58:40.341628 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8003eba-c8e6-4d54-acae-ac6c436694f2 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:58:40.341900 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8003eba-c8e6-4d54-acae-ac6c436694f2 None admin] PATH_INFO: `/v3/roles` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:58:40.342720 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8003eba-c8e6-4d54-acae-ac6c436694f2 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 21:58:40.343028 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8003eba-c8e6-4d54-acae-ac6c436694f2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:58:40.344447 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8003eba-c8e6-4d54-acae-ac6c436694f2 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:58:40.353628 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 92/193] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Jul 13 21:58:40 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 21:58:40.354404 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8003eba-c8e6-4d54-acae-ac6c436694f2 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:58:40.421304 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-135d9060-8a0d-4aa6-ad6f-d0f4e3b18ca8 None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:58:40.455503 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65242833-ddf1-4c42-8408-23ef43729c30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:58:40.456587 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65242833-ddf1-4c42-8408-23ef43729c30 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:58:40.456859 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65242833-ddf1-4c42-8408-23ef43729c30 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:58:40.457131 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65242833-ddf1-4c42-8408-23ef43729c30 None admin] PATH_INFO: `/v3/users/3b66c85500f54bc088f91021d5e64666` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:58:40.458433 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65242833-ddf1-4c42-8408-23ef43729c30 None admin] RBAC: Authorizing `identity:get_user(user_id=3b66c85500f54bc088f91021d5e64666)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:58:40.459952 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65242833-ddf1-4c42-8408-23ef43729c30 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:58:40.461597 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 102/194] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Jul 13 21:58:40 2022] GET /identity/v3/users/3b66c85500f54bc088f91021d5e64666 => generated 289 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 21:58:40.461811 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8003eba-c8e6-4d54-acae-ac6c436694f2 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:58:40.462351 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65242833-ddf1-4c42-8408-23ef43729c30 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:58:40.495488 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd95e260-de80-45f6-a3fa-7bc306ddfec2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:58:40.496159 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd95e260-de80-45f6-a3fa-7bc306ddfec2 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:58:40.496433 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd95e260-de80-45f6-a3fa-7bc306ddfec2 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:58:40.496722 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd95e260-de80-45f6-a3fa-7bc306ddfec2 None admin] PATH_INFO: `/v3/roles` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:58:40.497476 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd95e260-de80-45f6-a3fa-7bc306ddfec2 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 21:58:40.497774 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd95e260-de80-45f6-a3fa-7bc306ddfec2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:58:40.499189 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd95e260-de80-45f6-a3fa-7bc306ddfec2 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:58:40.504979 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:58:40.506881 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 93/195] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Jul 13 21:58:40 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 21:58:40.507721 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd95e260-de80-45f6-a3fa-7bc306ddfec2 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:58:40.544889 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:58:40.565023 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65242833-ddf1-4c42-8408-23ef43729c30 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:58:40.599136 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e2fe52-10dc-4f14-bf94-4e53f9125c2b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:58:40.599948 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e2fe52-10dc-4f14-bf94-4e53f9125c2b None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:58:40.600229 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e2fe52-10dc-4f14-bf94-4e53f9125c2b None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:58:40.600504 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9e2fe52-10dc-4f14-bf94-4e53f9125c2b None admin] PATH_INFO: `/v3/domains/default` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:58:40.601767 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e2fe52-10dc-4f14-bf94-4e53f9125c2b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:58:40.603286 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9e2fe52-10dc-4f14-bf94-4e53f9125c2b None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:58:40.605011 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 103/196] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Jul 13 21:58:40 2022] GET /identity/v3/domains/default => generated 199 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 21:58:40.605788 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e2fe52-10dc-4f14-bf94-4e53f9125c2b None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:58:40.611163 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd95e260-de80-45f6-a3fa-7bc306ddfec2 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:58:40.644846 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df25ecc-a93f-4213-b847-34051a3da111 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:58:40.645729 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df25ecc-a93f-4213-b847-34051a3da111 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:58:40.646006 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df25ecc-a93f-4213-b847-34051a3da111 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:58:40.646278 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4df25ecc-a93f-4213-b847-34051a3da111 None admin] PATH_INFO: `/v3/users/31bcb7d1181e4fb9b6a16d8729a9fee2` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:58:40.662516 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df25ecc-a93f-4213-b847-34051a3da111 None admin] RBAC: Authorizing `identity:get_user(user_id=31bcb7d1181e4fb9b6a16d8729a9fee2)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:58:40.663982 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4df25ecc-a93f-4213-b847-34051a3da111 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:58:40.665675 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 94/197] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Jul 13 21:58:40 2022] GET /identity/v3/users/31bcb7d1181e4fb9b6a16d8729a9fee2 => generated 283 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 21:58:40.666776 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df25ecc-a93f-4213-b847-34051a3da111 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:58:40.707274 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9e2fe52-10dc-4f14-bf94-4e53f9125c2b None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:58:40.717800 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:58:40.741951 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de303706-1a61-459b-8ebb-9c9883fa496b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:58:40.742636 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de303706-1a61-459b-8ebb-9c9883fa496b None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:58:40.742920 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de303706-1a61-459b-8ebb-9c9883fa496b None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:58:40.743194 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de303706-1a61-459b-8ebb-9c9883fa496b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:58:40.761068 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:58:40.768670 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4df25ecc-a93f-4213-b847-34051a3da111 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:58:40.803164 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3822284-a1a1-4054-88e1-3cc8e7b43fe8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:58:40.803976 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3822284-a1a1-4054-88e1-3cc8e7b43fe8 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:58:40.804256 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3822284-a1a1-4054-88e1-3cc8e7b43fe8 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:58:40.804527 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3822284-a1a1-4054-88e1-3cc8e7b43fe8 None admin] PATH_INFO: `/v3/users/31bcb7d1181e4fb9b6a16d8729a9fee2` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:58:40.805745 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3822284-a1a1-4054-88e1-3cc8e7b43fe8 None admin] RBAC: Authorizing `identity:get_user(user_id=31bcb7d1181e4fb9b6a16d8729a9fee2)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:58:40.807368 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3822284-a1a1-4054-88e1-3cc8e7b43fe8 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:58:40.809063 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 95/198] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Jul 13 21:58:40 2022] GET /identity/v3/users/31bcb7d1181e4fb9b6a16d8729a9fee2 => generated 283 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 21:58:40.809791 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3822284-a1a1-4054-88e1-3cc8e7b43fe8 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:58:40.812588 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de303706-1a61-459b-8ebb-9c9883fa496b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:58:40.814062 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de303706-1a61-459b-8ebb-9c9883fa496b None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:58:40.912100 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3822284-a1a1-4054-88e1-3cc8e7b43fe8 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:58:40.916917 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 104/199] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Jul 13 21:58:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 311 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 13 21:58:40.917734 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de303706-1a61-459b-8ebb-9c9883fa496b None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:58:40.929134 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:58:40.947766 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4138e5b3-43f4-4841-a914-e2d6c3ebc380 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:58:40.948454 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4138e5b3-43f4-4841-a914-e2d6c3ebc380 None admin] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:58:40.948737 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4138e5b3-43f4-4841-a914-e2d6c3ebc380 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:58:40.949012 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4138e5b3-43f4-4841-a914-e2d6c3ebc380 None admin] PATH_INFO: `/v3/users` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:58:40.950086 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4138e5b3-43f4-4841-a914-e2d6c3ebc380 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:58:40.951587 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4138e5b3-43f4-4841-a914-e2d6c3ebc380 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:58:40.971892 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 96/200] 10.222.0.52 () {60 vars in 1132 bytes} [Wed Jul 13 21:58:40 2022] POST /identity/v3/users => generated 297 bytes in 162 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 21:58:40.972702 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4138e5b3-43f4-4841-a914-e2d6c3ebc380 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:58:40.995351 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:58:41.030418 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de303706-1a61-459b-8ebb-9c9883fa496b None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:58:41.070222 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6608e24b-bdfe-4645-8832-e97f73845f74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:58:41.071017 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6608e24b-bdfe-4645-8832-e97f73845f74 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:58:41.071343 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6608e24b-bdfe-4645-8832-e97f73845f74 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:58:41.071610 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6608e24b-bdfe-4645-8832-e97f73845f74 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:58:41.090396 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4138e5b3-43f4-4841-a914-e2d6c3ebc380 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:58:41.130402 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91bb65ae-77e3-4089-8ddd-c5d93665608a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:58:41.131092 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bb65ae-77e3-4089-8ddd-c5d93665608a None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:58:41.131371 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bb65ae-77e3-4089-8ddd-c5d93665608a None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:58:41.131636 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91bb65ae-77e3-4089-8ddd-c5d93665608a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:58:41.151491 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6608e24b-bdfe-4645-8832-e97f73845f74 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:58:41.153004 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6608e24b-bdfe-4645-8832-e97f73845f74 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:58:41.167922 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:58:41.215733 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91bb65ae-77e3-4089-8ddd-c5d93665608a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:58:41.217228 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91bb65ae-77e3-4089-8ddd-c5d93665608a None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:58:41.244426 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:58:41.276872 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 105/201] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Jul 13 21:58:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 359 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 13 21:58:41.277772 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6608e24b-bdfe-4645-8832-e97f73845f74 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:58:41.346592 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 97/202] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Jul 13 21:58:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 13 21:58:41.347822 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91bb65ae-77e3-4089-8ddd-c5d93665608a None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:58:41.398693 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6608e24b-bdfe-4645-8832-e97f73845f74 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:58:41.414153 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:58:41.444958 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edcc2ef4-c855-4fde-b699-c92bc86710ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:58:41.446177 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcc2ef4-c855-4fde-b699-c92bc86710ce None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:58:41.446692 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcc2ef4-c855-4fde-b699-c92bc86710ce None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:58:41.447158 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-edcc2ef4-c855-4fde-b699-c92bc86710ce None admin] PATH_INFO: `/v3/projects/97aa9f72c7a54c9daa7e48288b4aab0f` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:58:41.448702 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edcc2ef4-c855-4fde-b699-c92bc86710ce None admin] RBAC: Authorizing `identity:get_project(project_id=97aa9f72c7a54c9daa7e48288b4aab0f)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:58:41.450882 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-edcc2ef4-c855-4fde-b699-c92bc86710ce None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:58:41.453256 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 106/203] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Jul 13 21:58:41 2022] GET /identity/v3/projects/97aa9f72c7a54c9daa7e48288b4aab0f => generated 298 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 21:58:41.458306 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91bb65ae-77e3-4089-8ddd-c5d93665608a None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:58:41.459360 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edcc2ef4-c855-4fde-b699-c92bc86710ce None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:58:41.480215 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:58:41.502536 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f934cd-d16a-4cc4-9d44-931a7a181e94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:58:41.503434 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f934cd-d16a-4cc4-9d44-931a7a181e94 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:58:41.503713 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f934cd-d16a-4cc4-9d44-931a7a181e94 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:58:41.503978 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65f934cd-d16a-4cc4-9d44-931a7a181e94 None admin] PATH_INFO: `/v3/projects/4896b13c732d49d09947c2baaf9dc04d` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:58:41.505533 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f934cd-d16a-4cc4-9d44-931a7a181e94 None admin] RBAC: Authorizing `identity:get_project(project_id=4896b13c732d49d09947c2baaf9dc04d)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:58:41.507397 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65f934cd-d16a-4cc4-9d44-931a7a181e94 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:58:41.509835 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 98/204] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Jul 13 21:58:41 2022] GET /identity/v3/projects/4896b13c732d49d09947c2baaf9dc04d => generated 302 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 21:58:41.510798 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f934cd-d16a-4cc4-9d44-931a7a181e94 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:58:41.576034 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-edcc2ef4-c855-4fde-b699-c92bc86710ce None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:58:41.619303 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf554ac-8cec-4484-acb3-48918aa61f51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:58:41.620461 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf554ac-8cec-4484-acb3-48918aa61f51 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:58:41.621057 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf554ac-8cec-4484-acb3-48918aa61f51 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:58:41.621337 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bf554ac-8cec-4484-acb3-48918aa61f51 None admin] PATH_INFO: `/v3/projects/4896b13c732d49d09947c2baaf9dc04d` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:58:41.622983 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf554ac-8cec-4484-acb3-48918aa61f51 None admin] RBAC: Authorizing `identity:get_project(project_id=4896b13c732d49d09947c2baaf9dc04d)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:58:41.625011 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bf554ac-8cec-4484-acb3-48918aa61f51 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:58:41.627474 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 107/205] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Jul 13 21:58:41 2022] GET /identity/v3/projects/4896b13c732d49d09947c2baaf9dc04d => generated 302 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 21:58:41.628250 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf554ac-8cec-4484-acb3-48918aa61f51 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:58:41.633995 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65f934cd-d16a-4cc4-9d44-931a7a181e94 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:58:41.669970 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7dc305-7e9e-4f4a-9533-4dcfba947433 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:58:41.670679 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7dc305-7e9e-4f4a-9533-4dcfba947433 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:58:41.670957 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7dc305-7e9e-4f4a-9533-4dcfba947433 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:58:41.671229 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d7dc305-7e9e-4f4a-9533-4dcfba947433 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:58:41.672123 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7dc305-7e9e-4f4a-9533-4dcfba947433 None admin] RBAC: Adding query filter params (role.id=abb8511cb7d54b249e85bbc7507f0317, scope.project.id=97aa9f72c7a54c9daa7e48288b4aab0f, user.id=3b66c85500f54bc088f91021d5e64666) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 21:58:41.672432 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7dc305-7e9e-4f4a-9533-4dcfba947433 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:58:41.673891 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d7dc305-7e9e-4f4a-9533-4dcfba947433 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:58:41.680165 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 99/206] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Jul 13 21:58:41 2022] GET /identity/v3/role_assignments?user.id=3b66c85500f54bc088f91021d5e64666&scope.project.id=97aa9f72c7a54c9daa7e48288b4aab0f&role.id=abb8511cb7d54b249e85bbc7507f0317 => generated 608 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 21:58:41.681062 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7dc305-7e9e-4f4a-9533-4dcfba947433 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:58:41.686263 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:58:41.757473 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bf554ac-8cec-4484-acb3-48918aa61f51 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:58:41.766875 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:58:41.796860 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d7dc305-7e9e-4f4a-9533-4dcfba947433 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:58:41.803931 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be88a4cc-d320-4cac-b7c7-6d827074797e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:58:41.804606 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be88a4cc-d320-4cac-b7c7-6d827074797e None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:58:41.804883 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be88a4cc-d320-4cac-b7c7-6d827074797e None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:58:41.805785 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be88a4cc-d320-4cac-b7c7-6d827074797e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:58:41.806708 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be88a4cc-d320-4cac-b7c7-6d827074797e None admin] RBAC: Adding query filter params (role.id=44efba2579f342df8a0789c59d163ec3, scope.project.id=4896b13c732d49d09947c2baaf9dc04d, user.id=31bcb7d1181e4fb9b6a16d8729a9fee2) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 21:58:41.807020 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be88a4cc-d320-4cac-b7c7-6d827074797e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:58:41.809695 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be88a4cc-d320-4cac-b7c7-6d827074797e None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:58:41.817886 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 108/207] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Jul 13 21:58:41 2022] GET /identity/v3/role_assignments?user.id=31bcb7d1181e4fb9b6a16d8729a9fee2&scope.project.id=4896b13c732d49d09947c2baaf9dc04d&role.id=44efba2579f342df8a0789c59d163ec3 => generated 259 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 21:58:41.848453 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37205378-fcad-4848-b098-1e0768548fbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:58:41.849208 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37205378-fcad-4848-b098-1e0768548fbd None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:58:41.849489 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37205378-fcad-4848-b098-1e0768548fbd None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:58:41.849759 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37205378-fcad-4848-b098-1e0768548fbd None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:58:41.850680 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37205378-fcad-4848-b098-1e0768548fbd None admin] RBAC: Adding query filter params (role.id=db84c6c897534af1a60b9ce62532f383, scope.project.id=4896b13c732d49d09947c2baaf9dc04d, user.id=31bcb7d1181e4fb9b6a16d8729a9fee2) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 21:58:41.851044 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37205378-fcad-4848-b098-1e0768548fbd None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:58:41.852596 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-37205378-fcad-4848-b098-1e0768548fbd None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:58:41.859877 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 100/208] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Jul 13 21:58:41 2022] GET /identity/v3/role_assignments?user.id=31bcb7d1181e4fb9b6a16d8729a9fee2&scope.project.id=4896b13c732d49d09947c2baaf9dc04d&role.id=db84c6c897534af1a60b9ce62532f383 => generated 259 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 21:58:43.156969 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1731ada5-ab33-41d1-a61e-a084cf771e2c None None] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:58:43.157494 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1731ada5-ab33-41d1-a61e-a084cf771e2c None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:58:43.157600 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1731ada5-ab33-41d1-a61e-a084cf771e2c None None] PATH_INFO: `/` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:58:43.159201 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 109/209] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jul 13 21:58:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 21:58:43.164434 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c951aff6-44c5-4fd5-a035-c08f64b052c6 None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:58:43.165223 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c951aff6-44c5-4fd5-a035-c08f64b052c6 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:58:43.165698 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c951aff6-44c5-4fd5-a035-c08f64b052c6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:58:43.183190 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-c951aff6-44c5-4fd5-a035-c08f64b052c6 None None] MFA Rules not processed for user `cc99c93da5074cf5a48bc2aa9a92c5e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:58:43.184686 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-c951aff6-44c5-4fd5-a035-c08f64b052c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:58:43.253655 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 101/210] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jul 13 21:58:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 13 21:58:43.277277 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1731ada5-ab33-41d1-a61e-a084cf771e2c None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:58:43.362776 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0294ee-ab24-4774-8628-512ca246921e None None] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:58:43.363434 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0294ee-ab24-4774-8628-512ca246921e None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:58:43.363908 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0294ee-ab24-4774-8628-512ca246921e None None] PATH_INFO: `/` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:58:43.365509 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 102/211] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jul 13 21:58:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 13 21:58:43.370652 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b998c86b-fa94-4ada-a6f5-dcf4171a29f6 None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:58:43.371169 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b998c86b-fa94-4ada-a6f5-dcf4171a29f6 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:58:43.371656 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b998c86b-fa94-4ada-a6f5-dcf4171a29f6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:58:43.386166 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1731ada5-ab33-41d1-a61e-a084cf771e2c None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:58:43.390736 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-b998c86b-fa94-4ada-a6f5-dcf4171a29f6 None None] MFA Rules not processed for user `cc99c93da5074cf5a48bc2aa9a92c5e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:58:43.392142 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-b998c86b-fa94-4ada-a6f5-dcf4171a29f6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:58:43.397415 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:58:43.435892 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455b0257-602a-44d3-96fd-f5996a2140b4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:58:43.437672 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455b0257-602a-44d3-96fd-f5996a2140b4 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:58:43.438132 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455b0257-602a-44d3-96fd-f5996a2140b4 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:58:43.438622 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-455b0257-602a-44d3-96fd-f5996a2140b4 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:58:43.449787 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:58:43.455639 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-455b0257-602a-44d3-96fd-f5996a2140b4 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 21:58:43.468963 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application [None req-455b0257-602a-44d3-96fd-f5996a2140b4 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 13 21:58:43.468963 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 21:58:43.468963 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 21:58:43.468963 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 21:58:43.468963 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 21:58:43.468963 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 21:58:43.468963 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 21:58:43.468963 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 21:58:43.468963 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 21:58:43.468963 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 21:58:43.468963 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 21:58:43.468963 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 21:58:43.468963 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 21:58:43.468963 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 21:58:43.468963 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 21:58:43.468963 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 21:58:43.470124 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 21:58:43.470124 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 21:58:43.470124 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 21:58:43.470124 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 21:58:43.470124 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 21:58:43.470124 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 21:58:43.470124 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 21:58:43.470124 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 21:58:43.470124 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 21:58:43.470124 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 21:58:43.470124 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 21:58:43.470124 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application with Lock( Jul 13 21:58:43.470124 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 21:58:43.470124 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._enter() Jul 13 21:58:43.470124 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 21:58:43.470124 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 21:58:43.471181 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 21:58:43.471181 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.creator() Jul 13 21:58:43.471181 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 21:58:43.471181 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application created_value = creator( Jul 13 21:58:43.471181 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 21:58:43.471181 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 21:58:43.471181 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 21:58:43.471181 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 21:58:43.471181 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 21:58:43.471181 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 21:58:43.471181 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 13 21:58:43.471181 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Jul 13 21:58:43.472497 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 110/212] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Jul 13 21:58:43 2022] GET /identity/v3/roles/member => generated 84 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 13 21:58:43.477668 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455b0257-602a-44d3-96fd-f5996a2140b4 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:58:43.487137 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 103/213] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jul 13 21:58:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 13 21:58:43.513487 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b998c86b-fa94-4ada-a6f5-dcf4171a29f6 None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:58:43.585857 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-455b0257-602a-44d3-96fd-f5996a2140b4 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:58:43.621251 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89228342-b3ed-4b23-b36f-fd9ef717ce84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:58:43.621924 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89228342-b3ed-4b23-b36f-fd9ef717ce84 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:58:43.622435 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89228342-b3ed-4b23-b36f-fd9ef717ce84 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:58:43.622866 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89228342-b3ed-4b23-b36f-fd9ef717ce84 None admin] PATH_INFO: `/v3/roles` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:58:43.623790 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89228342-b3ed-4b23-b36f-fd9ef717ce84 None admin] RBAC: Adding query filter params (name=member) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 21:58:43.624257 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89228342-b3ed-4b23-b36f-fd9ef717ce84 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:58:43.625834 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89228342-b3ed-4b23-b36f-fd9ef717ce84 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:58:43.626080 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b998c86b-fa94-4ada-a6f5-dcf4171a29f6 None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:58:43.635321 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 111/214] 10.222.0.52 () {56 vars in 1101 bytes} [Wed Jul 13 21:58:43 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 21:58:43.641696 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89228342-b3ed-4b23-b36f-fd9ef717ce84 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:58:43.662271 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8b5338-18f6-4a42-b095-1c2e093999e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:58:43.663350 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8b5338-18f6-4a42-b095-1c2e093999e9 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:58:43.663797 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8b5338-18f6-4a42-b095-1c2e093999e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:58:43.664206 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8b5338-18f6-4a42-b095-1c2e093999e9 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:58:43.665621 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8b5338-18f6-4a42-b095-1c2e093999e9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:58:43.668616 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8b5338-18f6-4a42-b095-1c2e093999e9 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:58:43.675071 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 104/215] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Jul 13 21:58:43 2022] GET /identity/v3/domains/default => generated 199 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 21:58:43.676147 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8b5338-18f6-4a42-b095-1c2e093999e9 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:58:43.678249 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:58:43.723965 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:58:43.755467 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89228342-b3ed-4b23-b36f-fd9ef717ce84 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:58:43.789857 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8b5338-18f6-4a42-b095-1c2e093999e9 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:58:43.792843 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0e2027-42f7-4f86-bf6b-714ad6c0dbea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:58:43.794587 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0e2027-42f7-4f86-bf6b-714ad6c0dbea None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:58:43.794587 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0e2027-42f7-4f86-bf6b-714ad6c0dbea None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:58:43.794587 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c0e2027-42f7-4f86-bf6b-714ad6c0dbea None admin] PATH_INFO: `/v3/users/1ec0bdf338274a62a01b54190aef9006` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:58:43.810136 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c0e2027-42f7-4f86-bf6b-714ad6c0dbea None admin] RBAC: Authorizing `identity:get_user(user_id=1ec0bdf338274a62a01b54190aef9006)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:58:43.811880 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c0e2027-42f7-4f86-bf6b-714ad6c0dbea None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:58:43.818275 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 112/216] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Jul 13 21:58:43 2022] GET /identity/v3/users/1ec0bdf338274a62a01b54190aef9006 => generated 297 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 21:58:43.819719 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0e2027-42f7-4f86-bf6b-714ad6c0dbea None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:58:43.831608 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5192728a-c6e6-4b3b-be88-531bad5de11f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:58:43.832559 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5192728a-c6e6-4b3b-be88-531bad5de11f None admin] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:58:43.833012 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5192728a-c6e6-4b3b-be88-531bad5de11f None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:58:43.833481 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5192728a-c6e6-4b3b-be88-531bad5de11f None admin] PATH_INFO: `/v3/users` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:58:43.834378 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5192728a-c6e6-4b3b-be88-531bad5de11f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:58:43.836078 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5192728a-c6e6-4b3b-be88-531bad5de11f None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:58:43.865072 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 105/217] 10.222.0.52 () {60 vars in 1132 bytes} [Wed Jul 13 21:58:43 2022] POST /identity/v3/users => generated 297 bytes in 189 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 21:58:43.928979 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed6152c-4d61-4892-a2ce-ebd53e84be97 None None] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:58:43.929278 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed6152c-4d61-4892-a2ce-ebd53e84be97 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:58:43.929657 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eed6152c-4d61-4892-a2ce-ebd53e84be97 None None] PATH_INFO: `/` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:58:43.930453 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:58:43.931569 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 106/218] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jul 13 21:58:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 13 21:58:43.938077 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7087a78-9645-455c-93b3-b318c357d296 None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:58:43.938391 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7087a78-9645-455c-93b3-b318c357d296 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:58:43.938651 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7087a78-9645-455c-93b3-b318c357d296 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:58:43.948969 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c0e2027-42f7-4f86-bf6b-714ad6c0dbea None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:58:43.958037 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-e7087a78-9645-455c-93b3-b318c357d296 None None] MFA Rules not processed for user `cc99c93da5074cf5a48bc2aa9a92c5e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:58:43.959465 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-e7087a78-9645-455c-93b3-b318c357d296 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:58:43.989884 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff70ec7-ce75-4451-bbb5-5e34676d05c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:58:43.990470 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff70ec7-ce75-4451-bbb5-5e34676d05c6 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:58:43.990776 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff70ec7-ce75-4451-bbb5-5e34676d05c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:58:43.991047 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ff70ec7-ce75-4451-bbb5-5e34676d05c6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:58:44.040920 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 107/219] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jul 13 21:58:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 13 21:58:44.040920 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8aa3ae-fce8-4901-bd26-d2d6b055fbd8 None None] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:58:44.040920 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8aa3ae-fce8-4901-bd26-d2d6b055fbd8 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:58:44.040920 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8aa3ae-fce8-4901-bd26-d2d6b055fbd8 None None] PATH_INFO: `/` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:58:44.040920 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 108/220] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jul 13 21:58:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 21:58:44.048415 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d7cba9-3a06-4b24-bf5b-fc73d0c594fe None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:58:44.048820 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d7cba9-3a06-4b24-bf5b-fc73d0c594fe None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:58:44.049183 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0d7cba9-3a06-4b24-bf5b-fc73d0c594fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:58:44.067529 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-b0d7cba9-3a06-4b24-bf5b-fc73d0c594fe None None] MFA Rules not processed for user `cc99c93da5074cf5a48bc2aa9a92c5e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:58:44.069458 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-b0d7cba9-3a06-4b24-bf5b-fc73d0c594fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:58:44.070331 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff70ec7-ce75-4451-bbb5-5e34676d05c6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:58:44.071840 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ff70ec7-ce75-4451-bbb5-5e34676d05c6 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:58:44.099761 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:58:44.151975 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 109/221] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jul 13 21:58:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 13 21:58:44.152352 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d7cba9-3a06-4b24-bf5b-fc73d0c594fe None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:58:44.178297 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:58:44.195413 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 113/222] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Jul 13 21:58:43 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 375 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 13 21:58:44.195709 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff70ec7-ce75-4451-bbb5-5e34676d05c6 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:58:44.279752 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0d7cba9-3a06-4b24-bf5b-fc73d0c594fe None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:58:44.319844 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ff70ec7-ce75-4451-bbb5-5e34676d05c6 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:58:44.327971 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1c37dd-42db-47e3-950c-7cec7980b91a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:58:44.329042 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1c37dd-42db-47e3-950c-7cec7980b91a None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:58:44.329511 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1c37dd-42db-47e3-950c-7cec7980b91a None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:58:44.330003 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1c37dd-42db-47e3-950c-7cec7980b91a None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:58:44.344073 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db1c37dd-42db-47e3-950c-7cec7980b91a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 21:58:44.354835 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application [None req-db1c37dd-42db-47e3-950c-7cec7980b91a None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 13 21:58:44.354835 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 21:58:44.354835 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 21:58:44.354835 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 21:58:44.354835 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 21:58:44.354835 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 21:58:44.354835 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 21:58:44.354835 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 21:58:44.354835 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 21:58:44.354835 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 21:58:44.354835 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 21:58:44.354835 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 21:58:44.354835 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 21:58:44.354835 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 21:58:44.354835 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 21:58:44.354835 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 21:58:44.355765 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 21:58:44.355765 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 21:58:44.355765 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 21:58:44.355765 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 21:58:44.355765 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 21:58:44.355765 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 21:58:44.355765 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 21:58:44.355765 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 21:58:44.355765 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 21:58:44.355765 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 21:58:44.355765 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 21:58:44.355765 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application with Lock( Jul 13 21:58:44.355765 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 21:58:44.355765 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._enter() Jul 13 21:58:44.355765 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 21:58:44.355765 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 21:58:44.356572 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 21:58:44.356572 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.creator() Jul 13 21:58:44.356572 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 21:58:44.356572 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application created_value = creator( Jul 13 21:58:44.356572 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 21:58:44.356572 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 21:58:44.356572 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 21:58:44.356572 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 21:58:44.356572 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 21:58:44.356572 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 21:58:44.356572 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 13 21:58:44.356572 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Jul 13 21:58:44.360259 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 110/223] 10.222.0.52 () {56 vars in 1096 bytes} [Wed Jul 13 21:58:44 2022] GET /identity/v3/roles/admin => generated 83 bytes in 208 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 21:58:44.361946 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1c37dd-42db-47e3-950c-7cec7980b91a None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:58:44.367047 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6184b030-1428-4250-9a9f-6cb9d3a84902 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:58:44.371434 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6184b030-1428-4250-9a9f-6cb9d3a84902 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:58:44.371721 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6184b030-1428-4250-9a9f-6cb9d3a84902 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:58:44.371999 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6184b030-1428-4250-9a9f-6cb9d3a84902 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:58:44.386666 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:58:44.386944 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6184b030-1428-4250-9a9f-6cb9d3a84902 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 21:58:44.400006 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application [None req-6184b030-1428-4250-9a9f-6cb9d3a84902 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 13 21:58:44.400006 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 21:58:44.400006 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 21:58:44.400006 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 21:58:44.400006 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 21:58:44.400006 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 21:58:44.400006 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 21:58:44.400006 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 21:58:44.400006 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 21:58:44.400006 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 21:58:44.400006 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 21:58:44.400006 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 21:58:44.400006 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 21:58:44.400006 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 21:58:44.400006 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 21:58:44.400006 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 21:58:44.401005 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 21:58:44.401005 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 21:58:44.401005 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 21:58:44.401005 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 21:58:44.401005 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 21:58:44.401005 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 21:58:44.401005 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 21:58:44.401005 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 21:58:44.401005 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 21:58:44.401005 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 21:58:44.401005 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 21:58:44.401005 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application with Lock( Jul 13 21:58:44.401005 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 21:58:44.401005 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._enter() Jul 13 21:58:44.401005 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 21:58:44.401005 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 21:58:44.401855 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 21:58:44.401855 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.creator() Jul 13 21:58:44.401855 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 21:58:44.401855 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application created_value = creator( Jul 13 21:58:44.401855 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 21:58:44.401855 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 21:58:44.401855 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 21:58:44.401855 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 21:58:44.401855 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 21:58:44.401855 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 21:58:44.401855 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 13 21:58:44.401855 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Jul 13 21:58:44.401855 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 114/224] 10.222.0.52 () {56 vars in 1114 bytes} [Wed Jul 13 21:58:44 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 21:58:44.402565 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6184b030-1428-4250-9a9f-6cb9d3a84902 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:58:44.460233 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:58:44.487402 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1c37dd-42db-47e3-950c-7cec7980b91a None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:58:44.515984 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6184b030-1428-4250-9a9f-6cb9d3a84902 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:58:44.524291 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9380df2d-1fcd-4d40-86c4-d5b6ee02a5be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:58:44.525179 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9380df2d-1fcd-4d40-86c4-d5b6ee02a5be None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:58:44.525462 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9380df2d-1fcd-4d40-86c4-d5b6ee02a5be None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:58:44.525734 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9380df2d-1fcd-4d40-86c4-d5b6ee02a5be None admin] PATH_INFO: `/v3/projects/4896b13c732d49d09947c2baaf9dc04d` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:58:44.527029 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9380df2d-1fcd-4d40-86c4-d5b6ee02a5be None admin] RBAC: Authorizing `identity:get_project(project_id=4896b13c732d49d09947c2baaf9dc04d)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:58:44.528572 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9380df2d-1fcd-4d40-86c4-d5b6ee02a5be None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:58:44.530361 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 111/225] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Jul 13 21:58:44 2022] GET /identity/v3/projects/4896b13c732d49d09947c2baaf9dc04d => generated 302 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 21:58:44.531095 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9380df2d-1fcd-4d40-86c4-d5b6ee02a5be None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:58:44.554869 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89aa4e84-3361-4d4b-9b6b-833eb83fd7e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:58:44.555708 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89aa4e84-3361-4d4b-9b6b-833eb83fd7e0 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:58:44.555864 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89aa4e84-3361-4d4b-9b6b-833eb83fd7e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:58:44.556140 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89aa4e84-3361-4d4b-9b6b-833eb83fd7e0 None admin] PATH_INFO: `/v3/roles` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:58:44.556956 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89aa4e84-3361-4d4b-9b6b-833eb83fd7e0 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 21:58:44.557373 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89aa4e84-3361-4d4b-9b6b-833eb83fd7e0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:58:44.558802 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89aa4e84-3361-4d4b-9b6b-833eb83fd7e0 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:58:44.567989 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 115/226] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Jul 13 21:58:44 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 21:58:44.568828 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89aa4e84-3361-4d4b-9b6b-833eb83fd7e0 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:58:44.627058 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:58:44.644120 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9380df2d-1fcd-4d40-86c4-d5b6ee02a5be None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:58:44.673881 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89aa4e84-3361-4d4b-9b6b-833eb83fd7e0 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:58:44.679596 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fc6647-42a7-4563-b45b-330ee1c2b32f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:58:44.680485 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fc6647-42a7-4563-b45b-330ee1c2b32f None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:58:44.680911 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fc6647-42a7-4563-b45b-330ee1c2b32f None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:58:44.681327 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2fc6647-42a7-4563-b45b-330ee1c2b32f None admin] PATH_INFO: `/v3/roles` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:58:44.682563 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2fc6647-42a7-4563-b45b-330ee1c2b32f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 21:58:44.683022 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2fc6647-42a7-4563-b45b-330ee1c2b32f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:58:44.684581 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2fc6647-42a7-4563-b45b-330ee1c2b32f None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:58:44.687335 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:58:44.693643 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 112/227] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Jul 13 21:58:44 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 21:58:44.694860 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fc6647-42a7-4563-b45b-330ee1c2b32f None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:58:44.713102 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6034ede1-ebd4-4a88-98e9-2a595a996156 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:58:44.713835 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6034ede1-ebd4-4a88-98e9-2a595a996156 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:58:44.714167 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6034ede1-ebd4-4a88-98e9-2a595a996156 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:58:44.714542 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6034ede1-ebd4-4a88-98e9-2a595a996156 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:58:44.715502 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6034ede1-ebd4-4a88-98e9-2a595a996156 None admin] RBAC: Adding query filter params (role.id=2a5e86976394498abb6310697cdb3f8b, scope.project.id=4896b13c732d49d09947c2baaf9dc04d, user.id=1ec0bdf338274a62a01b54190aef9006) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 21:58:44.715818 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6034ede1-ebd4-4a88-98e9-2a595a996156 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:58:44.717328 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6034ede1-ebd4-4a88-98e9-2a595a996156 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:58:44.723603 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 116/228] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Jul 13 21:58:44 2022] GET /identity/v3/role_assignments?user.id=1ec0bdf338274a62a01b54190aef9006&scope.project.id=4896b13c732d49d09947c2baaf9dc04d&role.id=2a5e86976394498abb6310697cdb3f8b => generated 259 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 21:58:44.724256 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6034ede1-ebd4-4a88-98e9-2a595a996156 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:58:44.807353 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2fc6647-42a7-4563-b45b-330ee1c2b32f None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:58:44.836174 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6034ede1-ebd4-4a88-98e9-2a595a996156 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:58:44.846064 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7997de-f3a0-46dc-9c10-45abb0055668 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:58:44.847434 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7997de-f3a0-46dc-9c10-45abb0055668 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:58:44.847916 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7997de-f3a0-46dc-9c10-45abb0055668 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:58:44.848383 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a7997de-f3a0-46dc-9c10-45abb0055668 None admin] PATH_INFO: `/v3/users/31bcb7d1181e4fb9b6a16d8729a9fee2` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:58:44.849896 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7997de-f3a0-46dc-9c10-45abb0055668 None admin] RBAC: Authorizing `identity:get_user(user_id=31bcb7d1181e4fb9b6a16d8729a9fee2)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:58:44.851878 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a7997de-f3a0-46dc-9c10-45abb0055668 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:58:44.853995 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 113/229] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Jul 13 21:58:44 2022] GET /identity/v3/users/31bcb7d1181e4fb9b6a16d8729a9fee2 => generated 283 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 21:58:44.861923 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7997de-f3a0-46dc-9c10-45abb0055668 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:58:44.882837 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc62e813-251f-4f37-9333-1a57bb9cad12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:58:44.883788 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc62e813-251f-4f37-9333-1a57bb9cad12 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:58:44.884063 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc62e813-251f-4f37-9333-1a57bb9cad12 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:58:44.884337 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc62e813-251f-4f37-9333-1a57bb9cad12 None admin] PATH_INFO: `/v3/users/31bcb7d1181e4fb9b6a16d8729a9fee2` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:58:44.887649 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc62e813-251f-4f37-9333-1a57bb9cad12 None admin] RBAC: Authorizing `identity:get_user(user_id=31bcb7d1181e4fb9b6a16d8729a9fee2)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:58:44.889364 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc62e813-251f-4f37-9333-1a57bb9cad12 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:58:44.894004 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 117/230] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Jul 13 21:58:44 2022] GET /identity/v3/users/31bcb7d1181e4fb9b6a16d8729a9fee2 => generated 283 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 21:58:44.899232 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc62e813-251f-4f37-9333-1a57bb9cad12 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:58:44.905294 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:58:44.969975 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:58:45.029073 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a7997de-f3a0-46dc-9c10-45abb0055668 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:58:45.080350 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc62e813-251f-4f37-9333-1a57bb9cad12 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:58:45.097510 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ebbb0b-eafb-48fc-9339-90461e972905 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:58:45.098397 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ebbb0b-eafb-48fc-9339-90461e972905 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:58:45.098707 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ebbb0b-eafb-48fc-9339-90461e972905 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:58:45.099036 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77ebbb0b-eafb-48fc-9339-90461e972905 None admin] PATH_INFO: `/v3/projects/4896b13c732d49d09947c2baaf9dc04d` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:58:45.100704 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ebbb0b-eafb-48fc-9339-90461e972905 None admin] RBAC: Authorizing `identity:get_project(project_id=4896b13c732d49d09947c2baaf9dc04d)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:58:45.102951 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77ebbb0b-eafb-48fc-9339-90461e972905 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:58:45.109116 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 114/231] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Jul 13 21:58:44 2022] GET /identity/v3/projects/4896b13c732d49d09947c2baaf9dc04d => generated 302 bytes in 247 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 21:58:45.110660 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ebbb0b-eafb-48fc-9339-90461e972905 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:58:45.154672 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c223af-5beb-441a-93c4-fb3d15fb3a6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:58:45.155560 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c223af-5beb-441a-93c4-fb3d15fb3a6f None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:58:45.155836 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c223af-5beb-441a-93c4-fb3d15fb3a6f None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:58:45.156164 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c223af-5beb-441a-93c4-fb3d15fb3a6f None admin] PATH_INFO: `/v3/projects/4896b13c732d49d09947c2baaf9dc04d` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:58:45.158002 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c223af-5beb-441a-93c4-fb3d15fb3a6f None admin] RBAC: Authorizing `identity:get_project(project_id=4896b13c732d49d09947c2baaf9dc04d)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:58:45.159717 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1c223af-5beb-441a-93c4-fb3d15fb3a6f None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:58:45.168878 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 118/232] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Jul 13 21:58:44 2022] GET /identity/v3/projects/4896b13c732d49d09947c2baaf9dc04d => generated 302 bytes in 265 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 21:58:45.170143 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c223af-5beb-441a-93c4-fb3d15fb3a6f None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:58:45.273203 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:58:45.305944 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77ebbb0b-eafb-48fc-9339-90461e972905 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:58:45.319386 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c223af-5beb-441a-93c4-fb3d15fb3a6f None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:58:45.333821 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:58:45.355163 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5c21fe-236c-43bb-aab1-2e1afdaa6b95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:58:45.356090 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5c21fe-236c-43bb-aab1-2e1afdaa6b95 None admin] REQUEST_METHOD: `PUT` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:58:45.356374 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5c21fe-236c-43bb-aab1-2e1afdaa6b95 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:58:45.356656 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5c21fe-236c-43bb-aab1-2e1afdaa6b95 None admin] PATH_INFO: `/v3/projects/4896b13c732d49d09947c2baaf9dc04d/users/31bcb7d1181e4fb9b6a16d8729a9fee2/roles/db84c6c897534af1a60b9ce62532f383` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:58:45.362938 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5c21fe-236c-43bb-aab1-2e1afdaa6b95 None admin] RBAC: Authorizing `identity:create_grant(project_id=4896b13c732d49d09947c2baaf9dc04d, user_id=31bcb7d1181e4fb9b6a16d8729a9fee2, role_id=db84c6c897534af1a60b9ce62532f383)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:58:45.365208 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5c21fe-236c-43bb-aab1-2e1afdaa6b95 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:58:45.367526 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-392a731f-eaab-4449-b547-82fa7711b5c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:58:45.368504 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392a731f-eaab-4449-b547-82fa7711b5c8 None admin] REQUEST_METHOD: `PUT` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:58:45.368850 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392a731f-eaab-4449-b547-82fa7711b5c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:58:45.369191 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-392a731f-eaab-4449-b547-82fa7711b5c8 None admin] PATH_INFO: `/v3/projects/4896b13c732d49d09947c2baaf9dc04d/users/31bcb7d1181e4fb9b6a16d8729a9fee2/roles/44efba2579f342df8a0789c59d163ec3` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:58:45.372757 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-392a731f-eaab-4449-b547-82fa7711b5c8 None admin] RBAC: Authorizing `identity:create_grant(project_id=4896b13c732d49d09947c2baaf9dc04d, user_id=31bcb7d1181e4fb9b6a16d8729a9fee2, role_id=44efba2579f342df8a0789c59d163ec3)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:58:45.374534 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-392a731f-eaab-4449-b547-82fa7711b5c8 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:58:45.390327 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 119/233] 10.222.0.52 () {58 vars in 1439 bytes} [Wed Jul 13 21:58:45 2022] PUT /identity/v3/projects/4896b13c732d49d09947c2baaf9dc04d/users/31bcb7d1181e4fb9b6a16d8729a9fee2/roles/db84c6c897534af1a60b9ce62532f383 => generated 0 bytes in 220 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 13 21:58:45.405131 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 115/234] 10.222.0.52 () {58 vars in 1439 bytes} [Wed Jul 13 21:58:45 2022] PUT /identity/v3/projects/4896b13c732d49d09947c2baaf9dc04d/users/31bcb7d1181e4fb9b6a16d8729a9fee2/roles/44efba2579f342df8a0789c59d163ec3 => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 21:58:46.346337 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d450c81b-5be4-4584-a299-1581f15e87cc None None] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:58:46.347442 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d450c81b-5be4-4584-a299-1581f15e87cc None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:58:46.347934 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d450c81b-5be4-4584-a299-1581f15e87cc None None] PATH_INFO: `/` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:58:46.349799 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 120/235] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jul 13 21:58:46 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 21:58:46.355236 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a2a78b-05e7-4cf3-a99a-d38f46de4994 None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:58:46.356055 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a2a78b-05e7-4cf3-a99a-d38f46de4994 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:58:46.356540 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7a2a78b-05e7-4cf3-a99a-d38f46de4994 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:58:46.375057 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-d7a2a78b-05e7-4cf3-a99a-d38f46de4994 None None] MFA Rules not processed for user `cc99c93da5074cf5a48bc2aa9a92c5e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:58:46.376612 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-d7a2a78b-05e7-4cf3-a99a-d38f46de4994 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:58:46.447237 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 116/236] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jul 13 21:58:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 13 21:58:46.471168 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d450c81b-5be4-4584-a299-1581f15e87cc None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:58:46.577819 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d450c81b-5be4-4584-a299-1581f15e87cc None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:58:46.590687 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38038f61-0b07-461f-bf89-21524a7e79ee None None] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:58:46.590976 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38038f61-0b07-461f-bf89-21524a7e79ee None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:58:46.591254 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38038f61-0b07-461f-bf89-21524a7e79ee None None] PATH_INFO: `/` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:58:46.592820 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 117/237] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jul 13 21:58:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 21:58:46.597570 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3347b4ff-ffab-42d0-a2d4-f21618d4a9c7 None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:58:46.597797 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3347b4ff-ffab-42d0-a2d4-f21618d4a9c7 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:58:46.598078 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3347b4ff-ffab-42d0-a2d4-f21618d4a9c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:58:46.617906 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16d65e1-b985-43ff-9c6f-6d54af9a6af2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:58:46.618891 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16d65e1-b985-43ff-9c6f-6d54af9a6af2 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:58:46.619182 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16d65e1-b985-43ff-9c6f-6d54af9a6af2 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:58:46.619462 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c16d65e1-b985-43ff-9c6f-6d54af9a6af2 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:58:46.626629 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-3347b4ff-ffab-42d0-a2d4-f21618d4a9c7 None None] MFA Rules not processed for user `cc99c93da5074cf5a48bc2aa9a92c5e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:58:46.628023 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-3347b4ff-ffab-42d0-a2d4-f21618d4a9c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:58:46.630839 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c16d65e1-b985-43ff-9c6f-6d54af9a6af2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 21:58:46.639750 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application [None req-c16d65e1-b985-43ff-9c6f-6d54af9a6af2 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 13 21:58:46.639750 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 21:58:46.639750 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 21:58:46.639750 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 21:58:46.639750 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 21:58:46.639750 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 21:58:46.639750 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 21:58:46.639750 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 21:58:46.639750 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 21:58:46.639750 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 21:58:46.639750 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 21:58:46.639750 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 21:58:46.639750 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 21:58:46.639750 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 21:58:46.639750 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 21:58:46.639750 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 21:58:46.640642 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 21:58:46.640642 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 21:58:46.640642 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 21:58:46.640642 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 21:58:46.640642 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 21:58:46.640642 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 21:58:46.640642 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 21:58:46.640642 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 21:58:46.640642 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 21:58:46.640642 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 21:58:46.640642 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 21:58:46.640642 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application with Lock( Jul 13 21:58:46.640642 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 21:58:46.640642 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._enter() Jul 13 21:58:46.640642 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 21:58:46.640642 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 21:58:46.641429 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 21:58:46.641429 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.creator() Jul 13 21:58:46.641429 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 21:58:46.641429 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application created_value = creator( Jul 13 21:58:46.641429 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 21:58:46.641429 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 21:58:46.641429 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 21:58:46.641429 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 21:58:46.641429 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 21:58:46.641429 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 21:58:46.641429 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 13 21:58:46.641429 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Jul 13 21:58:46.642021 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 121/238] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Jul 13 21:58:46 2022] GET /identity/v3/roles/reader => generated 84 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 21:58:46.648095 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16d65e1-b985-43ff-9c6f-6d54af9a6af2 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:58:46.683791 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:58:46.759455 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 118/239] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jul 13 21:58:46 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 13 21:58:46.764615 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c16d65e1-b985-43ff-9c6f-6d54af9a6af2 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:58:46.794099 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3347b4ff-ffab-42d0-a2d4-f21618d4a9c7 None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:58:46.797302 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:58:46.802407 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ad2f3c-fbc4-4680-985c-ff5e23a9a1c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:58:46.803121 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ad2f3c-fbc4-4680-985c-ff5e23a9a1c6 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:58:46.803401 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ad2f3c-fbc4-4680-985c-ff5e23a9a1c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:58:46.803678 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41ad2f3c-fbc4-4680-985c-ff5e23a9a1c6 None admin] PATH_INFO: `/v3/roles` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:58:46.804543 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ad2f3c-fbc4-4680-985c-ff5e23a9a1c6 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 21:58:46.804853 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ad2f3c-fbc4-4680-985c-ff5e23a9a1c6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:58:46.806335 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41ad2f3c-fbc4-4680-985c-ff5e23a9a1c6 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:58:46.818681 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 122/240] 10.222.0.52 () {56 vars in 1101 bytes} [Wed Jul 13 21:58:46 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 21:58:46.826185 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ad2f3c-fbc4-4680-985c-ff5e23a9a1c6 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:58:46.926161 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:58:46.934135 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41ad2f3c-fbc4-4680-985c-ff5e23a9a1c6 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:58:46.971789 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085f6b2f-2100-493a-b08e-67bb11e0a842 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:58:46.972960 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085f6b2f-2100-493a-b08e-67bb11e0a842 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:58:46.973386 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085f6b2f-2100-493a-b08e-67bb11e0a842 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:58:46.973802 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085f6b2f-2100-493a-b08e-67bb11e0a842 None admin] PATH_INFO: `/v3/users/286300b3ff1541089d0295569ac25679` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:58:46.988612 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085f6b2f-2100-493a-b08e-67bb11e0a842 None admin] RBAC: Authorizing `identity:get_user(user_id=286300b3ff1541089d0295569ac25679)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:58:46.990389 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085f6b2f-2100-493a-b08e-67bb11e0a842 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:58:46.992669 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 123/241] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Jul 13 21:58:46 2022] GET /identity/v3/users/286300b3ff1541089d0295569ac25679 => generated 297 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 21:58:46.998318 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085f6b2f-2100-493a-b08e-67bb11e0a842 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:58:47.034180 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3347b4ff-ffab-42d0-a2d4-f21618d4a9c7 None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:58:47.105975 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085f6b2f-2100-493a-b08e-67bb11e0a842 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:58:47.112500 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ede4de-088e-4813-aedf-f4bb1cd51d94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:58:47.113316 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ede4de-088e-4813-aedf-f4bb1cd51d94 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:58:47.113591 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ede4de-088e-4813-aedf-f4bb1cd51d94 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:58:47.113861 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23ede4de-088e-4813-aedf-f4bb1cd51d94 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:58:47.119377 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ede4de-088e-4813-aedf-f4bb1cd51d94 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:58:47.120742 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23ede4de-088e-4813-aedf-f4bb1cd51d94 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:58:47.122729 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 119/242] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Jul 13 21:58:46 2022] GET /identity/v3/domains/default => generated 199 bytes in 329 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 21:58:47.127661 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ede4de-088e-4813-aedf-f4bb1cd51d94 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:58:47.145007 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eec6831-ca7c-4253-b90d-5b9e441b75ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:58:47.145999 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eec6831-ca7c-4253-b90d-5b9e441b75ef None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:58:47.146477 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eec6831-ca7c-4253-b90d-5b9e441b75ef None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:58:47.146991 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4eec6831-ca7c-4253-b90d-5b9e441b75ef None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:58:47.170527 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:58:47.227510 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eec6831-ca7c-4253-b90d-5b9e441b75ef None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:58:47.228984 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4eec6831-ca7c-4253-b90d-5b9e441b75ef None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:58:47.328111 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:58:47.345018 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 124/243] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Jul 13 21:58:46 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 13 21:58:47.350668 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eec6831-ca7c-4253-b90d-5b9e441b75ef None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:58:47.381912 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23ede4de-088e-4813-aedf-f4bb1cd51d94 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:58:47.419797 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:58:47.429604 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09e8b877-5458-4ee8-b208-e559d341365d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:58:47.430919 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e8b877-5458-4ee8-b208-e559d341365d None admin] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:58:47.431205 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e8b877-5458-4ee8-b208-e559d341365d None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:58:47.431767 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09e8b877-5458-4ee8-b208-e559d341365d None admin] PATH_INFO: `/v3/users` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:58:47.432537 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e8b877-5458-4ee8-b208-e559d341365d None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:58:47.434359 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09e8b877-5458-4ee8-b208-e559d341365d None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:58:47.462782 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 120/244] 10.222.0.52 () {60 vars in 1132 bytes} [Wed Jul 13 21:58:47 2022] POST /identity/v3/users => generated 293 bytes in 335 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 13 21:58:47.464464 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031b0ccd-e8e9-4832-bb7d-d3b8126f4654 None None] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:58:47.464754 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031b0ccd-e8e9-4832-bb7d-d3b8126f4654 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:58:47.465031 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-031b0ccd-e8e9-4832-bb7d-d3b8126f4654 None None] PATH_INFO: `/` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:58:47.466502 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 121/245] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jul 13 21:58:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 21:58:47.473398 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cf6a7d-977f-42f0-8953-47f083d12c5c None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:58:47.473826 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cf6a7d-977f-42f0-8953-47f083d12c5c None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:58:47.474193 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91cf6a7d-977f-42f0-8953-47f083d12c5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:58:47.494227 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4eec6831-ca7c-4253-b90d-5b9e441b75ef None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:58:47.515679 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-91cf6a7d-977f-42f0-8953-47f083d12c5c None None] MFA Rules not processed for user `cc99c93da5074cf5a48bc2aa9a92c5e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:58:47.516900 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-91cf6a7d-977f-42f0-8953-47f083d12c5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:58:47.538822 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016e3fc3-cbd0-4244-bc74-297a2002eb3a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:58:47.539723 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016e3fc3-cbd0-4244-bc74-297a2002eb3a None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:58:47.540002 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016e3fc3-cbd0-4244-bc74-297a2002eb3a None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:58:47.540279 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-016e3fc3-cbd0-4244-bc74-297a2002eb3a None admin] PATH_INFO: `/v3/projects/4896b13c732d49d09947c2baaf9dc04d` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:58:47.541735 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016e3fc3-cbd0-4244-bc74-297a2002eb3a None admin] RBAC: Authorizing `identity:get_project(project_id=4896b13c732d49d09947c2baaf9dc04d)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:58:47.543552 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-016e3fc3-cbd0-4244-bc74-297a2002eb3a None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:58:47.545806 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 125/246] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Jul 13 21:58:47 2022] GET /identity/v3/projects/4896b13c732d49d09947c2baaf9dc04d => generated 302 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 21:58:47.556185 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016e3fc3-cbd0-4244-bc74-297a2002eb3a None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:58:47.676607 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-016e3fc3-cbd0-4244-bc74-297a2002eb3a None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:58:47.701864 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:58:47.716323 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 122/247] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jul 13 21:58:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 244 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 13 21:58:47.722082 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-848653b1-b8fd-4eba-8479-3a16cb9d1755 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:58:47.723062 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848653b1-b8fd-4eba-8479-3a16cb9d1755 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:58:47.723347 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848653b1-b8fd-4eba-8479-3a16cb9d1755 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:58:47.723623 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-848653b1-b8fd-4eba-8479-3a16cb9d1755 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:58:47.724482 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848653b1-b8fd-4eba-8479-3a16cb9d1755 None admin] RBAC: Adding query filter params (role.id=abb8511cb7d54b249e85bbc7507f0317, scope.project.id=4896b13c732d49d09947c2baaf9dc04d, user.id=286300b3ff1541089d0295569ac25679) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 21:58:47.724799 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848653b1-b8fd-4eba-8479-3a16cb9d1755 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:58:47.726264 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-848653b1-b8fd-4eba-8479-3a16cb9d1755 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:58:47.734387 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 126/248] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Jul 13 21:58:47 2022] GET /identity/v3/role_assignments?user.id=286300b3ff1541089d0295569ac25679&scope.project.id=4896b13c732d49d09947c2baaf9dc04d&role.id=abb8511cb7d54b249e85bbc7507f0317 => generated 259 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 21:58:47.740945 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91cf6a7d-977f-42f0-8953-47f083d12c5c None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:58:47.796282 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7153e1a1-5907-46c3-818d-78ba8986ef5b None None] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:58:47.796523 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7153e1a1-5907-46c3-818d-78ba8986ef5b None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:58:47.796755 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7153e1a1-5907-46c3-818d-78ba8986ef5b None None] PATH_INFO: `/` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:58:47.798160 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 127/249] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jul 13 21:58:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 21:58:47.804273 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a85d99-b472-4e17-bc99-fd1de8578951 None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:58:47.804568 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a85d99-b472-4e17-bc99-fd1de8578951 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:58:47.804823 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6a85d99-b472-4e17-bc99-fd1de8578951 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:58:47.827748 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-a6a85d99-b472-4e17-bc99-fd1de8578951 None None] MFA Rules not processed for user `cc99c93da5074cf5a48bc2aa9a92c5e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:58:47.828293 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-a6a85d99-b472-4e17-bc99-fd1de8578951 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:58:47.838897 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:58:47.873804 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91cf6a7d-977f-42f0-8953-47f083d12c5c None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:58:47.918116 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e11856-c578-4ea2-a7d2-ad189cccb5ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:58:47.919359 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e11856-c578-4ea2-a7d2-ad189cccb5ee None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:58:47.919468 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 128/250] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jul 13 21:58:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 13 21:58:47.919961 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e11856-c578-4ea2-a7d2-ad189cccb5ee None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:58:47.926509 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e11856-c578-4ea2-a7d2-ad189cccb5ee None admin] PATH_INFO: `/v3/roles/member` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:58:47.934510 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7fbab0-4b75-495a-9c33-245405e9ab51 None None] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:58:47.934510 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7fbab0-4b75-495a-9c33-245405e9ab51 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:58:47.934510 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb7fbab0-4b75-495a-9c33-245405e9ab51 None None] PATH_INFO: `/` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:58:47.934510 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 129/251] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jul 13 21:58:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 21:58:47.939015 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5f9b94-cedf-4108-bfee-84b94ff7a736 None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:58:47.939354 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5f9b94-cedf-4108-bfee-84b94ff7a736 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:58:47.939711 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b5f9b94-cedf-4108-bfee-84b94ff7a736 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:58:47.946058 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81e11856-c578-4ea2-a7d2-ad189cccb5ee None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 21:58:47.954858 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application [None req-81e11856-c578-4ea2-a7d2-ad189cccb5ee None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 13 21:58:47.954858 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 21:58:47.954858 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 21:58:47.954858 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 21:58:47.954858 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 21:58:47.954858 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 21:58:47.954858 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 21:58:47.954858 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 21:58:47.954858 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 21:58:47.954858 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 21:58:47.954858 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 21:58:47.954858 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 21:58:47.954858 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 21:58:47.954858 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 21:58:47.954858 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 21:58:47.954858 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 21:58:47.956025 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 21:58:47.956025 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 21:58:47.956025 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 21:58:47.956025 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 21:58:47.956025 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 21:58:47.956025 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 21:58:47.956025 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 21:58:47.956025 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 21:58:47.956025 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 21:58:47.956025 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 21:58:47.956025 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 21:58:47.956025 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application with Lock( Jul 13 21:58:47.956025 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 21:58:47.956025 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._enter() Jul 13 21:58:47.956025 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 21:58:47.956025 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 21:58:47.956897 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 21:58:47.956897 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.creator() Jul 13 21:58:47.956897 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 21:58:47.956897 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application created_value = creator( Jul 13 21:58:47.956897 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 21:58:47.956897 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 21:58:47.956897 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 21:58:47.956897 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 21:58:47.956897 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 21:58:47.956897 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 21:58:47.956897 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 13 21:58:47.956897 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Jul 13 21:58:47.957826 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 123/252] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Jul 13 21:58:47 2022] GET /identity/v3/roles/member => generated 84 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 21:58:47.958732 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e11856-c578-4ea2-a7d2-ad189cccb5ee None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:58:47.962354 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-1b5f9b94-cedf-4108-bfee-84b94ff7a736 None None] MFA Rules not processed for user `cc99c93da5074cf5a48bc2aa9a92c5e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:58:47.963923 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-1b5f9b94-cedf-4108-bfee-84b94ff7a736 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:58:48.044889 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 130/253] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jul 13 21:58:47 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 13 21:58:48.045768 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5f9b94-cedf-4108-bfee-84b94ff7a736 None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:58:48.086960 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81e11856-c578-4ea2-a7d2-ad189cccb5ee None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:58:48.106710 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:58:48.114443 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:58:48.137456 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9785bd81-2fc5-4c13-bf81-e084a1337a8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:58:48.138550 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9785bd81-2fc5-4c13-bf81-e084a1337a8d None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:58:48.138836 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9785bd81-2fc5-4c13-bf81-e084a1337a8d None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:58:48.139992 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9785bd81-2fc5-4c13-bf81-e084a1337a8d None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:58:48.153730 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9785bd81-2fc5-4c13-bf81-e084a1337a8d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 21:58:48.164186 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application [None req-9785bd81-2fc5-4c13-bf81-e084a1337a8d None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 13 21:58:48.164186 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 21:58:48.164186 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 21:58:48.164186 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 21:58:48.164186 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 21:58:48.164186 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 21:58:48.164186 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 21:58:48.164186 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 21:58:48.164186 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 21:58:48.164186 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 21:58:48.164186 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 21:58:48.164186 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 21:58:48.164186 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 21:58:48.164186 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 21:58:48.164186 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 21:58:48.164186 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 21:58:48.165135 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 21:58:48.165135 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 21:58:48.165135 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 21:58:48.165135 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 21:58:48.165135 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 21:58:48.165135 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 21:58:48.165135 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 21:58:48.165135 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 21:58:48.165135 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 21:58:48.165135 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 21:58:48.165135 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 21:58:48.165135 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application with Lock( Jul 13 21:58:48.165135 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 21:58:48.165135 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._enter() Jul 13 21:58:48.165135 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 21:58:48.165135 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 21:58:48.165953 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 21:58:48.165953 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.creator() Jul 13 21:58:48.165953 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 21:58:48.165953 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application created_value = creator( Jul 13 21:58:48.165953 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 21:58:48.165953 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 21:58:48.165953 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 21:58:48.165953 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 21:58:48.165953 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 21:58:48.165953 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 21:58:48.165953 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 13 21:58:48.165953 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Jul 13 21:58:48.165953 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 124/254] 10.222.0.52 () {56 vars in 1114 bytes} [Wed Jul 13 21:58:47 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 21:58:48.167200 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9785bd81-2fc5-4c13-bf81-e084a1337a8d None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:58:48.178556 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b5f9b94-cedf-4108-bfee-84b94ff7a736 None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:58:48.220085 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f603280-14db-411a-842e-e5ccd80cd8ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:58:48.220776 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f603280-14db-411a-842e-e5ccd80cd8ed None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:58:48.221210 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f603280-14db-411a-842e-e5ccd80cd8ed None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:58:48.221491 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f603280-14db-411a-842e-e5ccd80cd8ed None admin] PATH_INFO: `/v3/roles` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:58:48.222259 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f603280-14db-411a-842e-e5ccd80cd8ed None admin] RBAC: Adding query filter params (name=member) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 21:58:48.222697 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f603280-14db-411a-842e-e5ccd80cd8ed None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:58:48.224268 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f603280-14db-411a-842e-e5ccd80cd8ed None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:58:48.236419 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 131/255] 10.222.0.52 () {56 vars in 1101 bytes} [Wed Jul 13 21:58:48 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 21:58:48.237272 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f603280-14db-411a-842e-e5ccd80cd8ed None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:58:48.300005 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9785bd81-2fc5-4c13-bf81-e084a1337a8d None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:58:48.341530 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15ab464-086f-4ad8-a319-0dcf25876dd9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:58:48.342678 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15ab464-086f-4ad8-a319-0dcf25876dd9 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:58:48.343141 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15ab464-086f-4ad8-a319-0dcf25876dd9 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:58:48.343578 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b15ab464-086f-4ad8-a319-0dcf25876dd9 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:58:48.366738 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b15ab464-086f-4ad8-a319-0dcf25876dd9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 21:58:48.375091 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application [None req-b15ab464-086f-4ad8-a319-0dcf25876dd9 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 13 21:58:48.375091 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 21:58:48.375091 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 21:58:48.375091 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 21:58:48.375091 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 21:58:48.375091 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 21:58:48.375091 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 21:58:48.375091 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 21:58:48.375091 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 21:58:48.375091 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 21:58:48.375091 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 21:58:48.375091 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 21:58:48.375091 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 21:58:48.375091 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 21:58:48.375091 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 21:58:48.375091 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 21:58:48.375963 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 21:58:48.375963 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 21:58:48.375963 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 21:58:48.375963 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 21:58:48.375963 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 21:58:48.375963 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 21:58:48.375963 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 21:58:48.375963 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 21:58:48.375963 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 21:58:48.375963 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 21:58:48.375963 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 21:58:48.375963 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application with Lock( Jul 13 21:58:48.375963 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 21:58:48.375963 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._enter() Jul 13 21:58:48.375963 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 21:58:48.375963 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 21:58:48.376757 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 21:58:48.376757 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.creator() Jul 13 21:58:48.376757 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 21:58:48.376757 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application created_value = creator( Jul 13 21:58:48.376757 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 21:58:48.376757 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 21:58:48.376757 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 21:58:48.376757 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 21:58:48.376757 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 21:58:48.376757 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 21:58:48.376757 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 13 21:58:48.376757 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Jul 13 21:58:48.380455 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 125/256] 10.222.0.52 () {56 vars in 1096 bytes} [Wed Jul 13 21:58:48 2022] GET /identity/v3/roles/admin => generated 83 bytes in 214 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 21:58:48.381629 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15ab464-086f-4ad8-a319-0dcf25876dd9 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:58:48.435002 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:58:48.451002 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f603280-14db-411a-842e-e5ccd80cd8ed None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:58:48.464442 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:58:48.515850 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e01cc1f-9fd9-4b82-ad61-fe875ce22c1b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:58:48.516539 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e01cc1f-9fd9-4b82-ad61-fe875ce22c1b None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:58:48.516816 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e01cc1f-9fd9-4b82-ad61-fe875ce22c1b None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:58:48.517096 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e01cc1f-9fd9-4b82-ad61-fe875ce22c1b None admin] PATH_INFO: `/v3/roles` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:58:48.517862 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e01cc1f-9fd9-4b82-ad61-fe875ce22c1b None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 21:58:48.518169 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e01cc1f-9fd9-4b82-ad61-fe875ce22c1b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:58:48.519613 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e01cc1f-9fd9-4b82-ad61-fe875ce22c1b None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:58:48.534541 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b15ab464-086f-4ad8-a319-0dcf25876dd9 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:58:48.538355 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 132/257] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Jul 13 21:58:48 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 301 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 21:58:48.539279 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e01cc1f-9fd9-4b82-ad61-fe875ce22c1b None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:58:48.598027 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411508b5-8782-45b5-b8a2-ed1be54bc662 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:58:48.598866 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411508b5-8782-45b5-b8a2-ed1be54bc662 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:58:48.599147 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411508b5-8782-45b5-b8a2-ed1be54bc662 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:58:48.599419 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-411508b5-8782-45b5-b8a2-ed1be54bc662 None admin] PATH_INFO: `/v3/users/1ec0bdf338274a62a01b54190aef9006` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:58:48.600645 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411508b5-8782-45b5-b8a2-ed1be54bc662 None admin] RBAC: Authorizing `identity:get_user(user_id=1ec0bdf338274a62a01b54190aef9006)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:58:48.602302 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-411508b5-8782-45b5-b8a2-ed1be54bc662 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:58:48.608645 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 126/258] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Jul 13 21:58:48 2022] GET /identity/v3/users/1ec0bdf338274a62a01b54190aef9006 => generated 297 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 21:58:48.609551 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411508b5-8782-45b5-b8a2-ed1be54bc662 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:58:48.691272 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e01cc1f-9fd9-4b82-ad61-fe875ce22c1b None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:58:48.737840 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-411508b5-8782-45b5-b8a2-ed1be54bc662 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:58:48.738874 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043cc890-1e6d-4f71-82be-6c513fc3feea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:58:48.739784 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043cc890-1e6d-4f71-82be-6c513fc3feea None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:58:48.740251 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043cc890-1e6d-4f71-82be-6c513fc3feea None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:58:48.740711 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-043cc890-1e6d-4f71-82be-6c513fc3feea None admin] PATH_INFO: `/v3/roles` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:58:48.741690 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043cc890-1e6d-4f71-82be-6c513fc3feea None admin] RBAC: Adding query filter params (name=admin) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 21:58:48.742181 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043cc890-1e6d-4f71-82be-6c513fc3feea None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:58:48.743789 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-043cc890-1e6d-4f71-82be-6c513fc3feea None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:58:48.745152 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:58:48.761835 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 133/259] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Jul 13 21:58:48 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 21:58:48.762716 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043cc890-1e6d-4f71-82be-6c513fc3feea None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:58:48.783310 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20410394-8f3b-4160-930c-7a035fd5a37d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:58:48.784666 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20410394-8f3b-4160-930c-7a035fd5a37d None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:58:48.785141 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20410394-8f3b-4160-930c-7a035fd5a37d None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:58:48.785604 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-20410394-8f3b-4160-930c-7a035fd5a37d None admin] PATH_INFO: `/v3/users/31bcb7d1181e4fb9b6a16d8729a9fee2` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:58:48.793831 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20410394-8f3b-4160-930c-7a035fd5a37d None admin] RBAC: Authorizing `identity:get_user(user_id=31bcb7d1181e4fb9b6a16d8729a9fee2)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:58:48.798206 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-20410394-8f3b-4160-930c-7a035fd5a37d None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:58:48.800130 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 127/260] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Jul 13 21:58:48 2022] GET /identity/v3/users/31bcb7d1181e4fb9b6a16d8729a9fee2 => generated 283 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 21:58:48.802433 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20410394-8f3b-4160-930c-7a035fd5a37d None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:58:48.819652 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:58:48.941772 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-20410394-8f3b-4160-930c-7a035fd5a37d None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:58:48.979704 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef0a9fd-8655-4cbc-b20a-09ede4832dbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:58:48.980973 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef0a9fd-8655-4cbc-b20a-09ede4832dbf None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:58:48.981465 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef0a9fd-8655-4cbc-b20a-09ede4832dbf None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:58:48.981929 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aef0a9fd-8655-4cbc-b20a-09ede4832dbf None admin] PATH_INFO: `/v3/users/31bcb7d1181e4fb9b6a16d8729a9fee2` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:58:48.983412 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef0a9fd-8655-4cbc-b20a-09ede4832dbf None admin] RBAC: Authorizing `identity:get_user(user_id=31bcb7d1181e4fb9b6a16d8729a9fee2)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:58:48.985221 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aef0a9fd-8655-4cbc-b20a-09ede4832dbf None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:58:48.987288 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 128/261] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Jul 13 21:58:48 2022] GET /identity/v3/users/31bcb7d1181e4fb9b6a16d8729a9fee2 => generated 283 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 21:58:48.988365 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef0a9fd-8655-4cbc-b20a-09ede4832dbf None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:58:49.023749 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-043cc890-1e6d-4f71-82be-6c513fc3feea None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:58:49.046874 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:58:49.099105 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aef0a9fd-8655-4cbc-b20a-09ede4832dbf None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:58:49.128519 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbd7338-d1de-4d75-a441-a5ed1ff7f4f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:58:49.129439 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbd7338-d1de-4d75-a441-a5ed1ff7f4f8 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:58:49.129668 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbd7338-d1de-4d75-a441-a5ed1ff7f4f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:58:49.129941 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acbd7338-d1de-4d75-a441-a5ed1ff7f4f8 None admin] PATH_INFO: `/v3/projects/4896b13c732d49d09947c2baaf9dc04d` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:58:49.133575 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acbd7338-d1de-4d75-a441-a5ed1ff7f4f8 None admin] RBAC: Authorizing `identity:get_project(project_id=4896b13c732d49d09947c2baaf9dc04d)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:58:49.135225 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acbd7338-d1de-4d75-a441-a5ed1ff7f4f8 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:58:49.136136 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d359f4c3-afe9-4c3f-a3e3-70ebb9a8148c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:58:49.141636 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 134/262] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Jul 13 21:58:48 2022] GET /identity/v3/projects/4896b13c732d49d09947c2baaf9dc04d => generated 302 bytes in 375 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 21:58:49.141636 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbd7338-d1de-4d75-a441-a5ed1ff7f4f8 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:58:49.142063 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d359f4c3-afe9-4c3f-a3e3-70ebb9a8148c None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:58:49.142652 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d359f4c3-afe9-4c3f-a3e3-70ebb9a8148c None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:58:49.143212 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d359f4c3-afe9-4c3f-a3e3-70ebb9a8148c None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:58:49.264954 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d359f4c3-afe9-4c3f-a3e3-70ebb9a8148c None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:58:49.266800 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d359f4c3-afe9-4c3f-a3e3-70ebb9a8148c None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:58:49.330319 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:58:49.331909 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:58:49.338403 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acbd7338-d1de-4d75-a441-a5ed1ff7f4f8 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:58:49.405245 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47795ee6-2949-4fb2-869a-2a7448f8e74b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:58:49.405984 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47795ee6-2949-4fb2-869a-2a7448f8e74b None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:58:49.406417 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47795ee6-2949-4fb2-869a-2a7448f8e74b None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:58:49.406873 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47795ee6-2949-4fb2-869a-2a7448f8e74b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:58:49.415471 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 129/263] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Jul 13 21:58:48 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 427 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 13 21:58:49.416878 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d359f4c3-afe9-4c3f-a3e3-70ebb9a8148c None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:58:49.515905 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d359f4c3-afe9-4c3f-a3e3-70ebb9a8148c None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:58:49.542825 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47795ee6-2949-4fb2-869a-2a7448f8e74b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:58:49.544309 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47795ee6-2949-4fb2-869a-2a7448f8e74b None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:58:49.551046 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e162f3e2-3e0f-4961-9f39-622aee8fa62e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:58:49.552432 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e162f3e2-3e0f-4961-9f39-622aee8fa62e None admin] REQUEST_METHOD: `PUT` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:58:49.552898 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e162f3e2-3e0f-4961-9f39-622aee8fa62e None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:58:49.553377 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e162f3e2-3e0f-4961-9f39-622aee8fa62e None admin] PATH_INFO: `/v3/projects/4896b13c732d49d09947c2baaf9dc04d/users/1ec0bdf338274a62a01b54190aef9006/roles/2a5e86976394498abb6310697cdb3f8b` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:58:49.555633 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e162f3e2-3e0f-4961-9f39-622aee8fa62e None admin] RBAC: Authorizing `identity:create_grant(project_id=4896b13c732d49d09947c2baaf9dc04d, user_id=1ec0bdf338274a62a01b54190aef9006, role_id=2a5e86976394498abb6310697cdb3f8b)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:58:49.557309 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e162f3e2-3e0f-4961-9f39-622aee8fa62e None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:58:49.597387 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 130/264] 10.222.0.52 () {58 vars in 1439 bytes} [Wed Jul 13 21:58:49 2022] PUT /identity/v3/projects/4896b13c732d49d09947c2baaf9dc04d/users/1ec0bdf338274a62a01b54190aef9006/roles/2a5e86976394498abb6310697cdb3f8b => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 21:58:49.602796 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e162f3e2-3e0f-4961-9f39-622aee8fa62e None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:58:49.606482 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:58:49.685645 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:58:49.694372 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 135/265] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Jul 13 21:58:49 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 557 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 13 21:58:49.703618 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47795ee6-2949-4fb2-869a-2a7448f8e74b None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:58:49.768508 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e162f3e2-3e0f-4961-9f39-622aee8fa62e None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:58:49.849120 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0a94f8-0b46-4b3f-b11d-8e6f75411def None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:58:49.850006 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0a94f8-0b46-4b3f-b11d-8e6f75411def None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:58:49.850282 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0a94f8-0b46-4b3f-b11d-8e6f75411def None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:58:49.850575 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb0a94f8-0b46-4b3f-b11d-8e6f75411def None admin] PATH_INFO: `/v3/projects/4896b13c732d49d09947c2baaf9dc04d` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:58:49.853095 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0a94f8-0b46-4b3f-b11d-8e6f75411def None admin] RBAC: Authorizing `identity:get_project(project_id=4896b13c732d49d09947c2baaf9dc04d)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:58:49.854657 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb0a94f8-0b46-4b3f-b11d-8e6f75411def None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:58:49.856613 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 131/266] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Jul 13 21:58:49 2022] GET /identity/v3/projects/4896b13c732d49d09947c2baaf9dc04d => generated 302 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 21:58:49.862410 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0a94f8-0b46-4b3f-b11d-8e6f75411def None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:58:49.871572 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47795ee6-2949-4fb2-869a-2a7448f8e74b None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:58:49.934530 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e20422-cea5-41b0-977e-16e896fe2019 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:58:49.935790 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e20422-cea5-41b0-977e-16e896fe2019 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:58:49.936085 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e20422-cea5-41b0-977e-16e896fe2019 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:58:49.936366 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05e20422-cea5-41b0-977e-16e896fe2019 None admin] PATH_INFO: `/v3/projects/4896b13c732d49d09947c2baaf9dc04d` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:58:49.938026 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e20422-cea5-41b0-977e-16e896fe2019 None admin] RBAC: Authorizing `identity:get_project(project_id=4896b13c732d49d09947c2baaf9dc04d)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:58:49.939686 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05e20422-cea5-41b0-977e-16e896fe2019 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:58:49.942624 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 136/267] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Jul 13 21:58:49 2022] GET /identity/v3/projects/4896b13c732d49d09947c2baaf9dc04d => generated 302 bytes in 239 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 21:58:49.948440 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e20422-cea5-41b0-977e-16e896fe2019 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:58:49.969493 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:58:50.004230 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb0a94f8-0b46-4b3f-b11d-8e6f75411def None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:58:50.021293 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:58:50.044118 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3000a79c-2264-4da8-b8c5-6bab238bb933 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:58:50.044887 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3000a79c-2264-4da8-b8c5-6bab238bb933 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:58:50.045172 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3000a79c-2264-4da8-b8c5-6bab238bb933 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:58:50.045445 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3000a79c-2264-4da8-b8c5-6bab238bb933 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:58:50.046299 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3000a79c-2264-4da8-b8c5-6bab238bb933 None admin] RBAC: Adding query filter params (role.id=db84c6c897534af1a60b9ce62532f383, scope.project.id=4896b13c732d49d09947c2baaf9dc04d, user.id=31bcb7d1181e4fb9b6a16d8729a9fee2) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 21:58:50.046643 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3000a79c-2264-4da8-b8c5-6bab238bb933 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:58:50.048199 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3000a79c-2264-4da8-b8c5-6bab238bb933 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:58:50.060040 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05e20422-cea5-41b0-977e-16e896fe2019 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:58:50.066032 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 132/268] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Jul 13 21:58:49 2022] GET /identity/v3/role_assignments?user.id=31bcb7d1181e4fb9b6a16d8729a9fee2&scope.project.id=4896b13c732d49d09947c2baaf9dc04d&role.id=db84c6c897534af1a60b9ce62532f383 => generated 608 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 21:58:50.097499 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed84481c-1a99-4789-8a78-a7296375d23e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:58:50.098171 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed84481c-1a99-4789-8a78-a7296375d23e None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:58:50.098456 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed84481c-1a99-4789-8a78-a7296375d23e None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:58:50.098773 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed84481c-1a99-4789-8a78-a7296375d23e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:58:50.099632 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed84481c-1a99-4789-8a78-a7296375d23e None admin] RBAC: Adding query filter params (role.id=44efba2579f342df8a0789c59d163ec3, scope.project.id=4896b13c732d49d09947c2baaf9dc04d, user.id=31bcb7d1181e4fb9b6a16d8729a9fee2) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 21:58:50.099949 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed84481c-1a99-4789-8a78-a7296375d23e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:58:50.101437 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed84481c-1a99-4789-8a78-a7296375d23e None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:58:50.108372 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 137/269] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Jul 13 21:58:49 2022] GET /identity/v3/role_assignments?user.id=31bcb7d1181e4fb9b6a16d8729a9fee2&scope.project.id=4896b13c732d49d09947c2baaf9dc04d&role.id=44efba2579f342df8a0789c59d163ec3 => generated 608 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 21:58:50.256903 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972f3841-27cf-41c1-ae66-f55ab8ca4352 None None] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:58:50.257259 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972f3841-27cf-41c1-ae66-f55ab8ca4352 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:58:50.257647 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972f3841-27cf-41c1-ae66-f55ab8ca4352 None None] PATH_INFO: `/` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:58:50.259190 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 133/270] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jul 13 21:58:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 21:58:50.265542 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2493a080-f042-469e-b560-e5df3499a079 None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:58:50.265833 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2493a080-f042-469e-b560-e5df3499a079 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:58:50.266107 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2493a080-f042-469e-b560-e5df3499a079 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:58:50.288789 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-2493a080-f042-469e-b560-e5df3499a079 None None] MFA Rules not processed for user `cc99c93da5074cf5a48bc2aa9a92c5e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:58:50.289934 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-2493a080-f042-469e-b560-e5df3499a079 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:58:50.368316 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 138/271] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jul 13 21:58:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 13 21:58:50.390806 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972f3841-27cf-41c1-ae66-f55ab8ca4352 None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:58:50.467333 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d065b642-817e-4ac3-ac49-7b35545399b4 None None] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:58:50.467684 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d065b642-817e-4ac3-ac49-7b35545399b4 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:58:50.467887 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d065b642-817e-4ac3-ac49-7b35545399b4 None None] PATH_INFO: `/` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:58:50.469424 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 139/272] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jul 13 21:58:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 13 21:58:50.474889 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35855e04-9ce2-4791-ba17-e90fcabcf9fc None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:58:50.475188 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35855e04-9ce2-4791-ba17-e90fcabcf9fc None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:58:50.476134 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35855e04-9ce2-4791-ba17-e90fcabcf9fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:58:50.506828 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-35855e04-9ce2-4791-ba17-e90fcabcf9fc None None] MFA Rules not processed for user `cc99c93da5074cf5a48bc2aa9a92c5e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:58:50.507743 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972f3841-27cf-41c1-ae66-f55ab8ca4352 None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:58:50.508025 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-35855e04-9ce2-4791-ba17-e90fcabcf9fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:58:50.566668 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e26055-4c2e-4bd8-a2f2-93c33c79f1b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:58:50.567580 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e26055-4c2e-4bd8-a2f2-93c33c79f1b9 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:58:50.567883 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e26055-4c2e-4bd8-a2f2-93c33c79f1b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:58:50.568162 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e26055-4c2e-4bd8-a2f2-93c33c79f1b9 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:58:50.572040 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:58:50.590983 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e26055-4c2e-4bd8-a2f2-93c33c79f1b9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 21:58:50.598742 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application [None req-b3e26055-4c2e-4bd8-a2f2-93c33c79f1b9 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 13 21:58:50.598742 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 21:58:50.598742 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 21:58:50.598742 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 21:58:50.598742 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 21:58:50.598742 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 21:58:50.598742 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 21:58:50.598742 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 21:58:50.598742 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 21:58:50.598742 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 21:58:50.598742 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 21:58:50.598742 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 21:58:50.598742 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 21:58:50.598742 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 21:58:50.598742 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 21:58:50.598742 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 21:58:50.599673 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 21:58:50.599673 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 21:58:50.599673 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 21:58:50.599673 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 21:58:50.599673 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 21:58:50.599673 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 21:58:50.599673 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 21:58:50.599673 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 21:58:50.599673 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 21:58:50.599673 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 21:58:50.599673 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 21:58:50.599673 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application with Lock( Jul 13 21:58:50.599673 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 21:58:50.599673 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._enter() Jul 13 21:58:50.599673 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 21:58:50.599673 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 21:58:50.600486 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 21:58:50.600486 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.creator() Jul 13 21:58:50.600486 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 21:58:50.600486 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application created_value = creator( Jul 13 21:58:50.600486 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 21:58:50.600486 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 21:58:50.600486 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 21:58:50.600486 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 21:58:50.600486 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 21:58:50.600486 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 21:58:50.600486 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 13 21:58:50.600486 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Jul 13 21:58:50.600486 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 134/273] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Jul 13 21:58:50 2022] GET /identity/v3/roles/reader => generated 84 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 21:58:50.603704 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:58:50.616710 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e26055-4c2e-4bd8-a2f2-93c33c79f1b9 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:58:50.621619 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 140/274] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jul 13 21:58:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 13 21:58:50.645942 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35855e04-9ce2-4791-ba17-e90fcabcf9fc None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:58:50.734056 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e26055-4c2e-4bd8-a2f2-93c33c79f1b9 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:58:50.751357 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35855e04-9ce2-4791-ba17-e90fcabcf9fc None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:58:50.770472 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6232117-598a-474f-be51-fb8420edf313 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:58:50.771169 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6232117-598a-474f-be51-fb8420edf313 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:58:50.771446 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6232117-598a-474f-be51-fb8420edf313 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:58:50.771713 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6232117-598a-474f-be51-fb8420edf313 None admin] PATH_INFO: `/v3/roles` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:58:50.772870 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6232117-598a-474f-be51-fb8420edf313 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 21:58:50.773179 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6232117-598a-474f-be51-fb8420edf313 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:58:50.774595 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6232117-598a-474f-be51-fb8420edf313 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:58:50.784179 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 135/275] 10.222.0.52 () {56 vars in 1101 bytes} [Wed Jul 13 21:58:50 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 21:58:50.785278 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7b78e4-76fe-4731-ba6a-485cbe4024a8 None None] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:58:50.785563 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7b78e4-76fe-4731-ba6a-485cbe4024a8 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:58:50.785830 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e7b78e4-76fe-4731-ba6a-485cbe4024a8 None None] PATH_INFO: `/` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:58:50.787215 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 136/276] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jul 13 21:58:50 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 21:58:50.789399 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7b78e4-76fe-4731-ba6a-485cbe4024a8 None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:58:50.790060 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be080140-9814-46b6-9288-47e70ad9eee3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:58:50.791170 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be080140-9814-46b6-9288-47e70ad9eee3 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:58:50.791581 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be080140-9814-46b6-9288-47e70ad9eee3 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:58:50.791844 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be080140-9814-46b6-9288-47e70ad9eee3 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:58:50.805241 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be080140-9814-46b6-9288-47e70ad9eee3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 21:58:50.813121 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application [None req-be080140-9814-46b6-9288-47e70ad9eee3 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 13 21:58:50.813121 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 21:58:50.813121 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 21:58:50.813121 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 21:58:50.813121 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 21:58:50.813121 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 21:58:50.813121 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 21:58:50.813121 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 21:58:50.813121 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 21:58:50.813121 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 21:58:50.813121 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 21:58:50.813121 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 21:58:50.813121 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 21:58:50.813121 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 21:58:50.813121 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 21:58:50.813121 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 21:58:50.814010 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 21:58:50.814010 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 21:58:50.814010 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 21:58:50.814010 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 21:58:50.814010 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 21:58:50.814010 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 21:58:50.814010 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 21:58:50.814010 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 21:58:50.814010 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 21:58:50.814010 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 21:58:50.814010 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 21:58:50.814010 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application with Lock( Jul 13 21:58:50.814010 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 21:58:50.814010 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._enter() Jul 13 21:58:50.814010 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 21:58:50.814010 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 21:58:50.815684 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 21:58:50.815684 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.creator() Jul 13 21:58:50.815684 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 21:58:50.815684 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application created_value = creator( Jul 13 21:58:50.815684 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 21:58:50.815684 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 21:58:50.815684 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 21:58:50.815684 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 21:58:50.815684 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 21:58:50.815684 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 21:58:50.815684 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 13 21:58:50.815684 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Jul 13 21:58:50.815684 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 141/277] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Jul 13 21:58:50 2022] GET /identity/v3/roles/member => generated 84 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 13 21:58:50.815684 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f58af1-47ff-4a6d-9949-ec9aafe2a006 None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:58:50.816393 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f58af1-47ff-4a6d-9949-ec9aafe2a006 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:58:50.816393 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6f58af1-47ff-4a6d-9949-ec9aafe2a006 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:58:50.832456 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-c6f58af1-47ff-4a6d-9949-ec9aafe2a006 None None] MFA Rules not processed for user `cc99c93da5074cf5a48bc2aa9a92c5e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:58:50.833672 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-c6f58af1-47ff-4a6d-9949-ec9aafe2a006 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:58:50.858310 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:58:50.880507 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:58:50.904661 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 142/278] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jul 13 21:58:50 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 13 21:58:50.905992 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f58af1-47ff-4a6d-9949-ec9aafe2a006 None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:58:50.906831 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e7b78e4-76fe-4731-ba6a-485cbe4024a8 None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:58:50.946167 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a118cbb-1606-4c70-ad60-1bfa905e35bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:58:50.947089 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a118cbb-1606-4c70-ad60-1bfa905e35bb None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:58:50.947409 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a118cbb-1606-4c70-ad60-1bfa905e35bb None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:58:50.947880 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a118cbb-1606-4c70-ad60-1bfa905e35bb None admin] PATH_INFO: `/v3/users/286300b3ff1541089d0295569ac25679` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:58:50.949608 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a118cbb-1606-4c70-ad60-1bfa905e35bb None admin] RBAC: Authorizing `identity:get_user(user_id=286300b3ff1541089d0295569ac25679)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:58:50.951246 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a118cbb-1606-4c70-ad60-1bfa905e35bb None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:58:50.952901 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 137/279] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Jul 13 21:58:50 2022] GET /identity/v3/users/286300b3ff1541089d0295569ac25679 => generated 297 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 21:58:50.958481 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a118cbb-1606-4c70-ad60-1bfa905e35bb None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:58:51.027820 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6f58af1-47ff-4a6d-9949-ec9aafe2a006 None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:58:51.065176 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc85ba0-85ef-42cb-bbb9-93345f5300c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:58:51.065923 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc85ba0-85ef-42cb-bbb9-93345f5300c0 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:58:51.066201 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc85ba0-85ef-42cb-bbb9-93345f5300c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:58:51.066476 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbc85ba0-85ef-42cb-bbb9-93345f5300c0 None admin] PATH_INFO: `/v3/roles` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:58:51.067291 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc85ba0-85ef-42cb-bbb9-93345f5300c0 None admin] RBAC: Adding query filter params (name=member) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 21:58:51.067604 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc85ba0-85ef-42cb-bbb9-93345f5300c0 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:58:51.069039 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbc85ba0-85ef-42cb-bbb9-93345f5300c0 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:58:51.070133 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a118cbb-1606-4c70-ad60-1bfa905e35bb None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:58:51.082806 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 143/280] 10.222.0.52 () {56 vars in 1101 bytes} [Wed Jul 13 21:58:50 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 21:58:51.082806 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc85ba0-85ef-42cb-bbb9-93345f5300c0 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:58:51.104943 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:58:51.115328 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec88179b-54f6-426a-9663-a3eb9bafd643 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:58:51.116224 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec88179b-54f6-426a-9663-a3eb9bafd643 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:58:51.116598 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec88179b-54f6-426a-9663-a3eb9bafd643 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:58:51.116875 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec88179b-54f6-426a-9663-a3eb9bafd643 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:58:51.118055 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec88179b-54f6-426a-9663-a3eb9bafd643 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:58:51.119577 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec88179b-54f6-426a-9663-a3eb9bafd643 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:58:51.121221 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 138/281] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Jul 13 21:58:50 2022] GET /identity/v3/domains/default => generated 199 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 21:58:51.121972 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec88179b-54f6-426a-9663-a3eb9bafd643 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:58:51.141605 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:58:51.204225 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbc85ba0-85ef-42cb-bbb9-93345f5300c0 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:58:51.222089 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec88179b-54f6-426a-9663-a3eb9bafd643 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:58:51.240235 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a8bf6d-4ff4-4344-823f-9bd985a548fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:58:51.242274 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a8bf6d-4ff4-4344-823f-9bd985a548fd None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:58:51.243051 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a8bf6d-4ff4-4344-823f-9bd985a548fd None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:58:51.243949 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34a8bf6d-4ff4-4344-823f-9bd985a548fd None admin] PATH_INFO: `/v3/projects/4896b13c732d49d09947c2baaf9dc04d` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:58:51.246840 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a8bf6d-4ff4-4344-823f-9bd985a548fd None admin] RBAC: Authorizing `identity:get_project(project_id=4896b13c732d49d09947c2baaf9dc04d)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:58:51.249936 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34a8bf6d-4ff4-4344-823f-9bd985a548fd None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:58:51.252111 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 144/282] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Jul 13 21:58:51 2022] GET /identity/v3/projects/4896b13c732d49d09947c2baaf9dc04d => generated 302 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 21:58:51.253035 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a8bf6d-4ff4-4344-823f-9bd985a548fd None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:58:51.259924 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd15c817-f49d-4564-9c38-11a5a9db1727 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:58:51.261119 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd15c817-f49d-4564-9c38-11a5a9db1727 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:58:51.261278 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd15c817-f49d-4564-9c38-11a5a9db1727 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:58:51.261553 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd15c817-f49d-4564-9c38-11a5a9db1727 None admin] PATH_INFO: `/v3/users/2c1dc01ea50848449386ae75b0c5ece6` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:58:51.276612 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd15c817-f49d-4564-9c38-11a5a9db1727 None admin] RBAC: Authorizing `identity:get_user(user_id=2c1dc01ea50848449386ae75b0c5ece6)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:58:51.278578 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd15c817-f49d-4564-9c38-11a5a9db1727 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:58:51.280722 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 139/283] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Jul 13 21:58:51 2022] GET /identity/v3/users/2c1dc01ea50848449386ae75b0c5ece6 => generated 293 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 21:58:51.281640 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd15c817-f49d-4564-9c38-11a5a9db1727 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:58:51.337158 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:58:51.365736 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34a8bf6d-4ff4-4344-823f-9bd985a548fd None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:58:51.377388 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:58:51.391111 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd15c817-f49d-4564-9c38-11a5a9db1727 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:58:51.401751 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a0ac4b-6b06-4d18-9b81-e92f6c062603 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:58:51.402429 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a0ac4b-6b06-4d18-9b81-e92f6c062603 None admin] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:58:51.402740 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a0ac4b-6b06-4d18-9b81-e92f6c062603 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:58:51.403015 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a0ac4b-6b06-4d18-9b81-e92f6c062603 None admin] PATH_INFO: `/v3/users` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:58:51.403723 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a0ac4b-6b06-4d18-9b81-e92f6c062603 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:58:51.405176 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a0ac4b-6b06-4d18-9b81-e92f6c062603 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:58:51.427882 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c80bf624-636e-43c7-9f91-2175344c2682 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:58:51.428788 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80bf624-636e-43c7-9f91-2175344c2682 None admin] REQUEST_METHOD: `PUT` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:58:51.429068 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80bf624-636e-43c7-9f91-2175344c2682 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:58:51.429397 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c80bf624-636e-43c7-9f91-2175344c2682 None admin] PATH_INFO: `/v3/projects/4896b13c732d49d09947c2baaf9dc04d/users/286300b3ff1541089d0295569ac25679/roles/abb8511cb7d54b249e85bbc7507f0317` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:58:51.431534 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c80bf624-636e-43c7-9f91-2175344c2682 None admin] RBAC: Authorizing `identity:create_grant(project_id=4896b13c732d49d09947c2baaf9dc04d, user_id=286300b3ff1541089d0295569ac25679, role_id=abb8511cb7d54b249e85bbc7507f0317)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:58:51.433231 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 145/284] 10.222.0.52 () {60 vars in 1132 bytes} [Wed Jul 13 21:58:51 2022] POST /identity/v3/users => generated 293 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 21:58:51.433720 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c80bf624-636e-43c7-9f91-2175344c2682 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:58:51.444119 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a0ac4b-6b06-4d18-9b81-e92f6c062603 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:58:51.459247 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 140/285] 10.222.0.52 () {58 vars in 1439 bytes} [Wed Jul 13 21:58:51 2022] PUT /identity/v3/projects/4896b13c732d49d09947c2baaf9dc04d/users/286300b3ff1541089d0295569ac25679/roles/abb8511cb7d54b249e85bbc7507f0317 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 21:58:51.575929 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a0ac4b-6b06-4d18-9b81-e92f6c062603 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:58:51.617328 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a605858-1e88-4336-9bc6-8966b958c5df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:58:51.617945 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a605858-1e88-4336-9bc6-8966b958c5df None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:58:51.618198 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a605858-1e88-4336-9bc6-8966b958c5df None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:58:51.618474 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a605858-1e88-4336-9bc6-8966b958c5df None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:58:51.619585 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a605858-1e88-4336-9bc6-8966b958c5df None admin] RBAC: Adding query filter params (role.id=2a5e86976394498abb6310697cdb3f8b, user.id=2c1dc01ea50848449386ae75b0c5ece6, scope.system=all) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 21:58:51.621016 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a605858-1e88-4336-9bc6-8966b958c5df None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:58:51.623528 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a605858-1e88-4336-9bc6-8966b958c5df None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:58:51.629583 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 146/286] 10.222.0.52 () {56 vars in 1308 bytes} [Wed Jul 13 21:58:51 2022] GET /identity/v3/role_assignments?user.id=2c1dc01ea50848449386ae75b0c5ece6&scope.system=all&role.id=2a5e86976394498abb6310697cdb3f8b => generated 226 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 21:58:52.093312 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2d810d-f1ae-468d-9d5c-1a45b3683467 None None] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:58:52.093843 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2d810d-f1ae-468d-9d5c-1a45b3683467 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:58:52.093843 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b2d810d-f1ae-468d-9d5c-1a45b3683467 None None] PATH_INFO: `/` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:58:52.095539 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 141/287] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jul 13 21:58:52 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 21:58:52.100874 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e47ee44-8f13-4db8-9e19-322bf332ab92 None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:58:52.101159 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e47ee44-8f13-4db8-9e19-322bf332ab92 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:58:52.101437 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e47ee44-8f13-4db8-9e19-322bf332ab92 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:58:52.123489 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-7e47ee44-8f13-4db8-9e19-322bf332ab92 None None] MFA Rules not processed for user `cc99c93da5074cf5a48bc2aa9a92c5e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:58:52.124689 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-7e47ee44-8f13-4db8-9e19-322bf332ab92 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:58:52.159551 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:58:52.218527 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 147/288] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jul 13 21:58:52 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 13 21:58:52.246683 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2d810d-f1ae-468d-9d5c-1a45b3683467 None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:58:52.372299 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b2d810d-f1ae-468d-9d5c-1a45b3683467 None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:58:52.383856 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:58:52.420223 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e8529e-cbb2-453a-820e-e8d75350cdae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:58:52.421044 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e8529e-cbb2-453a-820e-e8d75350cdae None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:58:52.421325 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e8529e-cbb2-453a-820e-e8d75350cdae None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:58:52.421602 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e8529e-cbb2-453a-820e-e8d75350cdae None admin] PATH_INFO: `/v3/roles/member` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:58:52.436712 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e8529e-cbb2-453a-820e-e8d75350cdae None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 21:58:52.449001 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application [None req-d0e8529e-cbb2-453a-820e-e8d75350cdae None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 13 21:58:52.449001 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 21:58:52.449001 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 21:58:52.449001 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 21:58:52.449001 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 21:58:52.449001 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 21:58:52.449001 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 21:58:52.449001 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 21:58:52.449001 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 21:58:52.449001 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 21:58:52.449001 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 21:58:52.449001 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 21:58:52.449001 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 21:58:52.449001 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 21:58:52.449001 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 21:58:52.449001 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 21:58:52.450251 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 21:58:52.450251 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 21:58:52.450251 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 21:58:52.450251 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 21:58:52.450251 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 21:58:52.450251 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 21:58:52.450251 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 21:58:52.450251 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 21:58:52.450251 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 21:58:52.450251 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 21:58:52.450251 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 21:58:52.450251 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application with Lock( Jul 13 21:58:52.450251 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 21:58:52.450251 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._enter() Jul 13 21:58:52.450251 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 21:58:52.450251 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 21:58:52.451075 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 21:58:52.451075 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.creator() Jul 13 21:58:52.451075 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 21:58:52.451075 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application created_value = creator( Jul 13 21:58:52.451075 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 21:58:52.451075 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 21:58:52.451075 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 21:58:52.451075 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 21:58:52.451075 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 21:58:52.451075 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 21:58:52.451075 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 13 21:58:52.451075 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Jul 13 21:58:52.451075 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 142/289] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Jul 13 21:58:52 2022] GET /identity/v3/roles/member => generated 84 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 21:58:52.456679 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e47ee44-8f13-4db8-9e19-322bf332ab92 None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:58:52.625298 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e47ee44-8f13-4db8-9e19-322bf332ab92 None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:58:52.672445 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c92eee5-5103-4379-9917-33540e807e16 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:58:52.673122 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c92eee5-5103-4379-9917-33540e807e16 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:58:52.673404 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c92eee5-5103-4379-9917-33540e807e16 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:58:52.673686 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c92eee5-5103-4379-9917-33540e807e16 None admin] PATH_INFO: `/v3/roles` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:58:52.674445 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c92eee5-5103-4379-9917-33540e807e16 None admin] RBAC: Adding query filter params (name=member) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 21:58:52.674774 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c92eee5-5103-4379-9917-33540e807e16 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:58:52.676190 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c92eee5-5103-4379-9917-33540e807e16 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:58:52.689982 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 148/290] 10.222.0.52 () {56 vars in 1101 bytes} [Wed Jul 13 21:58:52 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 21:58:52.695355 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e8529e-cbb2-453a-820e-e8d75350cdae None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:58:52.872890 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e8529e-cbb2-453a-820e-e8d75350cdae None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:58:52.921927 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be2c51b-6c15-4de5-8f98-be7828b5c8c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:58:52.923208 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be2c51b-6c15-4de5-8f98-be7828b5c8c5 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:58:52.923679 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be2c51b-6c15-4de5-8f98-be7828b5c8c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:58:52.924158 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5be2c51b-6c15-4de5-8f98-be7828b5c8c5 None admin] PATH_INFO: `/v3/users/1ec0bdf338274a62a01b54190aef9006` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:58:52.925573 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be2c51b-6c15-4de5-8f98-be7828b5c8c5 None admin] RBAC: Authorizing `identity:get_user(user_id=1ec0bdf338274a62a01b54190aef9006)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:58:52.927243 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5be2c51b-6c15-4de5-8f98-be7828b5c8c5 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:58:52.929077 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 143/291] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Jul 13 21:58:52 2022] GET /identity/v3/users/1ec0bdf338274a62a01b54190aef9006 => generated 297 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 21:58:52.934679 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c92eee5-5103-4379-9917-33540e807e16 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:58:52.977742 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:58:53.088792 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c92eee5-5103-4379-9917-33540e807e16 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:58:53.137412 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c8a1a2-a84e-4122-8557-f4a8a47b9272 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:58:53.138433 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c8a1a2-a84e-4122-8557-f4a8a47b9272 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:58:53.138959 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c8a1a2-a84e-4122-8557-f4a8a47b9272 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:58:53.139418 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32c8a1a2-a84e-4122-8557-f4a8a47b9272 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:58:53.230169 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c8a1a2-a84e-4122-8557-f4a8a47b9272 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:58:53.232062 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32c8a1a2-a84e-4122-8557-f4a8a47b9272 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:58:53.264969 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:58:53.363093 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 149/292] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Jul 13 21:58:52 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 429 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 13 21:58:53.368696 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be2c51b-6c15-4de5-8f98-be7828b5c8c5 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:58:53.423417 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:58:53.492019 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5be2c51b-6c15-4de5-8f98-be7828b5c8c5 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:58:53.544256 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8035ae15-5438-4564-8ded-ef5716c0a60c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:58:53.545147 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8035ae15-5438-4564-8ded-ef5716c0a60c None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:58:53.545432 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8035ae15-5438-4564-8ded-ef5716c0a60c None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:58:53.545706 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8035ae15-5438-4564-8ded-ef5716c0a60c None admin] PATH_INFO: `/v3/projects/4896b13c732d49d09947c2baaf9dc04d` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:58:53.549410 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8035ae15-5438-4564-8ded-ef5716c0a60c None admin] RBAC: Authorizing `identity:get_project(project_id=4896b13c732d49d09947c2baaf9dc04d)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:58:53.549410 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8035ae15-5438-4564-8ded-ef5716c0a60c None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:58:53.550151 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 144/293] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Jul 13 21:58:53 2022] GET /identity/v3/projects/4896b13c732d49d09947c2baaf9dc04d => generated 302 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 21:58:53.559638 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c8a1a2-a84e-4122-8557-f4a8a47b9272 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:58:53.691017 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32c8a1a2-a84e-4122-8557-f4a8a47b9272 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:58:53.719070 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:58:53.729991 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4a1cf5-8717-4971-9b32-2e8abcdb2980 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:58:53.730699 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4a1cf5-8717-4971-9b32-2e8abcdb2980 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:58:53.730982 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4a1cf5-8717-4971-9b32-2e8abcdb2980 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:58:53.731262 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4a1cf5-8717-4971-9b32-2e8abcdb2980 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:58:53.732123 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4a1cf5-8717-4971-9b32-2e8abcdb2980 None admin] RBAC: Adding query filter params (role.id=2a5e86976394498abb6310697cdb3f8b, scope.project.id=4896b13c732d49d09947c2baaf9dc04d, user.id=1ec0bdf338274a62a01b54190aef9006) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 21:58:53.732447 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4a1cf5-8717-4971-9b32-2e8abcdb2980 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:58:53.740159 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4a1cf5-8717-4971-9b32-2e8abcdb2980 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:58:53.747460 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 150/294] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Jul 13 21:58:53 2022] GET /identity/v3/role_assignments?user.id=1ec0bdf338274a62a01b54190aef9006&scope.project.id=4896b13c732d49d09947c2baaf9dc04d&role.id=2a5e86976394498abb6310697cdb3f8b => generated 608 bytes in 188 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 21:58:53.988211 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53130f64-0e2c-4153-86ef-bfdf2af8e082 None None] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:58:53.988609 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53130f64-0e2c-4153-86ef-bfdf2af8e082 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:58:53.988776 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53130f64-0e2c-4153-86ef-bfdf2af8e082 None None] PATH_INFO: `/` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:58:53.990167 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 145/295] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jul 13 21:58:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 21:58:53.994975 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b3d1c3-5705-4cc4-bcaa-6a206bc389e6 None None] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:58:53.995267 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b3d1c3-5705-4cc4-bcaa-6a206bc389e6 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:58:53.995556 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4b3d1c3-5705-4cc4-bcaa-6a206bc389e6 None None] PATH_INFO: `/` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:58:53.997266 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 151/296] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jul 13 21:58:53 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 21:58:53.999137 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51af6c5-4298-47b7-8f88-8f88bbf0be73 None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:58:53.999371 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51af6c5-4298-47b7-8f88-8f88bbf0be73 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:58:53.999641 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f51af6c5-4298-47b7-8f88-8f88bbf0be73 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:58:54.005776 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fe2539-05a5-4440-8a0b-58887edfa931 None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:58:54.006405 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fe2539-05a5-4440-8a0b-58887edfa931 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:58:54.006910 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2fe2539-05a5-4440-8a0b-58887edfa931 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:58:54.024064 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-f51af6c5-4298-47b7-8f88-8f88bbf0be73 None None] MFA Rules not processed for user `cc99c93da5074cf5a48bc2aa9a92c5e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:58:54.025288 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-f51af6c5-4298-47b7-8f88-8f88bbf0be73 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:58:54.025736 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-b2fe2539-05a5-4440-8a0b-58887edfa931 None None] MFA Rules not processed for user `cc99c93da5074cf5a48bc2aa9a92c5e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:58:54.027289 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-b2fe2539-05a5-4440-8a0b-58887edfa931 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:58:54.107716 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 152/297] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jul 13 21:58:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 13 21:58:54.124430 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 146/298] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jul 13 21:58:53 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 13 21:58:54.134129 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2fe2539-05a5-4440-8a0b-58887edfa931 None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:58:54.156945 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51af6c5-4298-47b7-8f88-8f88bbf0be73 None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:58:54.202387 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:58:54.262291 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2fe2539-05a5-4440-8a0b-58887edfa931 None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:58:54.280982 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:58:54.283183 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f51af6c5-4298-47b7-8f88-8f88bbf0be73 None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:58:54.300223 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3a21692-8297-497d-9cf9-c7a0205285b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:58:54.301030 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a21692-8297-497d-9cf9-c7a0205285b5 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:58:54.301321 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a21692-8297-497d-9cf9-c7a0205285b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:58:54.301571 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3a21692-8297-497d-9cf9-c7a0205285b5 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:58:54.312615 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3a21692-8297-497d-9cf9-c7a0205285b5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 21:58:54.319915 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34c9f8b2-7cd0-4e82-9aab-61196e0efadc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:58:54.320281 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application [None req-c3a21692-8297-497d-9cf9-c7a0205285b5 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 13 21:58:54.320281 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 21:58:54.320281 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 21:58:54.320281 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 21:58:54.320281 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 21:58:54.320281 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 21:58:54.320281 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 21:58:54.320281 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 21:58:54.320281 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 21:58:54.320281 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 21:58:54.320281 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 21:58:54.320281 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 21:58:54.320281 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 21:58:54.320281 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 21:58:54.320281 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 21:58:54.320281 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 21:58:54.321252 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 21:58:54.321252 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 21:58:54.321252 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 21:58:54.321252 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 21:58:54.321252 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 21:58:54.321252 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 21:58:54.321252 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 21:58:54.321252 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 21:58:54.321252 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 21:58:54.321252 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 21:58:54.321252 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 21:58:54.321252 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application with Lock( Jul 13 21:58:54.321252 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 21:58:54.321252 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._enter() Jul 13 21:58:54.321252 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 21:58:54.321252 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 21:58:54.322064 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 21:58:54.322064 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.creator() Jul 13 21:58:54.322064 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 21:58:54.322064 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application created_value = creator( Jul 13 21:58:54.322064 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 21:58:54.322064 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 21:58:54.322064 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 21:58:54.322064 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 21:58:54.322064 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 21:58:54.322064 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 21:58:54.322064 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 13 21:58:54.322064 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Jul 13 21:58:54.322723 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c9f8b2-7cd0-4e82-9aab-61196e0efadc None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:58:54.322723 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c9f8b2-7cd0-4e82-9aab-61196e0efadc None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:58:54.322723 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34c9f8b2-7cd0-4e82-9aab-61196e0efadc None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:58:54.322905 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 153/299] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Jul 13 21:58:54 2022] GET /identity/v3/roles/member => generated 84 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 21:58:54.322905 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fe4db6-a9c0-43b2-b1c0-b926ff10b989 None None] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:58:54.323568 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fe4db6-a9c0-43b2-b1c0-b926ff10b989 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:58:54.323568 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88fe4db6-a9c0-43b2-b1c0-b926ff10b989 None None] PATH_INFO: `/` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:58:54.325124 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 154/300] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jul 13 21:58:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 21:58:54.329153 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5c11f0-0204-474c-ac92-537ee4550d07 None None] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:58:54.329153 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5c11f0-0204-474c-ac92-537ee4550d07 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:58:54.329153 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e5c11f0-0204-474c-ac92-537ee4550d07 None None] PATH_INFO: `/` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:58:54.329153 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 155/301] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jul 13 21:58:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 21:58:54.329416 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5c11f0-0204-474c-ac92-537ee4550d07 None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:58:54.334862 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34c9f8b2-7cd0-4e82-9aab-61196e0efadc None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 21:58:54.342998 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application [None req-34c9f8b2-7cd0-4e82-9aab-61196e0efadc None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 13 21:58:54.342998 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 21:58:54.342998 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 21:58:54.342998 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 21:58:54.342998 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 21:58:54.342998 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 21:58:54.342998 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 21:58:54.342998 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 21:58:54.342998 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 21:58:54.342998 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 21:58:54.342998 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 21:58:54.342998 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 21:58:54.342998 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 21:58:54.342998 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 21:58:54.342998 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 21:58:54.342998 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 21:58:54.343862 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 21:58:54.343862 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 21:58:54.343862 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 21:58:54.343862 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 21:58:54.343862 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 21:58:54.343862 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 21:58:54.343862 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 21:58:54.343862 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 21:58:54.343862 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 21:58:54.343862 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 21:58:54.343862 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 21:58:54.343862 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application with Lock( Jul 13 21:58:54.343862 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 21:58:54.343862 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._enter() Jul 13 21:58:54.343862 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 21:58:54.343862 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 21:58:54.344651 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 21:58:54.344651 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.creator() Jul 13 21:58:54.344651 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 21:58:54.344651 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application created_value = creator( Jul 13 21:58:54.344651 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 21:58:54.344651 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 21:58:54.344651 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 21:58:54.344651 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 21:58:54.344651 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 21:58:54.344651 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 21:58:54.344651 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 13 21:58:54.344651 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Jul 13 21:58:54.344651 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 147/302] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Jul 13 21:58:54 2022] GET /identity/v3/roles/reader => generated 84 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 13 21:58:54.345488 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb727af-b01f-4f95-9732-cac7ee16458b None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:58:54.345798 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb727af-b01f-4f95-9732-cac7ee16458b None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:58:54.346079 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebb727af-b01f-4f95-9732-cac7ee16458b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:58:54.362711 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-ebb727af-b01f-4f95-9732-cac7ee16458b None None] MFA Rules not processed for user `cc99c93da5074cf5a48bc2aa9a92c5e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:58:54.363904 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-ebb727af-b01f-4f95-9732-cac7ee16458b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:58:54.429586 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e5c11f0-0204-474c-ac92-537ee4550d07 None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:58:54.430631 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 148/303] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jul 13 21:58:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 13 21:58:54.432081 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08b7cfa-20b6-4944-ad7f-82819d282d3d None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:58:54.432405 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08b7cfa-20b6-4944-ad7f-82819d282d3d None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:58:54.432730 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b08b7cfa-20b6-4944-ad7f-82819d282d3d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:58:54.448754 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-b08b7cfa-20b6-4944-ad7f-82819d282d3d None None] MFA Rules not processed for user `cc99c93da5074cf5a48bc2aa9a92c5e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:58:54.450287 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-b08b7cfa-20b6-4944-ad7f-82819d282d3d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:58:54.470706 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:58:54.478376 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c2efa8-dc4a-49f3-976f-4b65ddb9a4a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:58:54.479113 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c2efa8-dc4a-49f3-976f-4b65ddb9a4a2 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:58:54.479500 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c2efa8-dc4a-49f3-976f-4b65ddb9a4a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:58:54.479776 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22c2efa8-dc4a-49f3-976f-4b65ddb9a4a2 None admin] PATH_INFO: `/v3/roles` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:58:54.480534 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c2efa8-dc4a-49f3-976f-4b65ddb9a4a2 None admin] RBAC: Adding query filter params (name=member) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 21:58:54.480834 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c2efa8-dc4a-49f3-976f-4b65ddb9a4a2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:58:54.482237 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22c2efa8-dc4a-49f3-976f-4b65ddb9a4a2 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:58:54.490869 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 156/304] 10.222.0.52 () {56 vars in 1101 bytes} [Wed Jul 13 21:58:54 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 21:58:54.496413 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c2efa8-dc4a-49f3-976f-4b65ddb9a4a2 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:58:54.530558 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 149/305] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jul 13 21:58:54 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 13 21:58:54.530558 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08b7cfa-20b6-4944-ad7f-82819d282d3d None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:58:54.541460 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:58:54.599811 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22c2efa8-dc4a-49f3-976f-4b65ddb9a4a2 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:58:54.616307 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b08b7cfa-20b6-4944-ad7f-82819d282d3d None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:58:54.635075 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304774f0-eaa9-40a7-9923-7cf5856beb23 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:58:54.635747 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304774f0-eaa9-40a7-9923-7cf5856beb23 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:58:54.636023 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304774f0-eaa9-40a7-9923-7cf5856beb23 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:58:54.636292 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-304774f0-eaa9-40a7-9923-7cf5856beb23 None admin] PATH_INFO: `/v3/roles` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:58:54.637037 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304774f0-eaa9-40a7-9923-7cf5856beb23 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 21:58:54.637336 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304774f0-eaa9-40a7-9923-7cf5856beb23 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:58:54.638771 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-304774f0-eaa9-40a7-9923-7cf5856beb23 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:58:54.646801 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 157/306] 10.222.0.52 () {56 vars in 1101 bytes} [Wed Jul 13 21:58:54 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 21:58:54.647603 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304774f0-eaa9-40a7-9923-7cf5856beb23 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:58:54.653737 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3429e2-b57d-4144-b380-449268fdcce3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:58:54.654581 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3429e2-b57d-4144-b380-449268fdcce3 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:58:54.654854 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3429e2-b57d-4144-b380-449268fdcce3 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:58:54.655119 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3429e2-b57d-4144-b380-449268fdcce3 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:58:54.665023 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3429e2-b57d-4144-b380-449268fdcce3 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 21:58:54.673129 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application [None req-2b3429e2-b57d-4144-b380-449268fdcce3 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 13 21:58:54.673129 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 21:58:54.673129 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 21:58:54.673129 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 21:58:54.673129 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 21:58:54.673129 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 21:58:54.673129 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 21:58:54.673129 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 21:58:54.673129 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 21:58:54.673129 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 21:58:54.673129 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 21:58:54.673129 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 21:58:54.673129 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 21:58:54.673129 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 21:58:54.673129 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 21:58:54.673129 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 21:58:54.674093 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 21:58:54.674093 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 21:58:54.674093 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 21:58:54.674093 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 21:58:54.674093 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 21:58:54.674093 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 21:58:54.674093 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 21:58:54.674093 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 21:58:54.674093 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 21:58:54.674093 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 21:58:54.674093 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 21:58:54.674093 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application with Lock( Jul 13 21:58:54.674093 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 21:58:54.674093 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._enter() Jul 13 21:58:54.674093 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 21:58:54.674093 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 21:58:54.680066 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 21:58:54.680066 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.creator() Jul 13 21:58:54.680066 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 21:58:54.680066 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application created_value = creator( Jul 13 21:58:54.680066 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 21:58:54.680066 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 21:58:54.680066 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 21:58:54.680066 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 21:58:54.680066 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 21:58:54.680066 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 21:58:54.680066 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 13 21:58:54.680066 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Jul 13 21:58:54.680066 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 150/307] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Jul 13 21:58:54 2022] GET /identity/v3/roles/reader => generated 84 bytes in 157 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 21:58:54.680066 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3429e2-b57d-4144-b380-449268fdcce3 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:58:54.696675 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:58:54.747707 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:58:54.751266 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-304774f0-eaa9-40a7-9923-7cf5856beb23 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:58:54.779246 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3429e2-b57d-4144-b380-449268fdcce3 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:58:54.787793 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f5e8df-4f3e-429d-b4b7-857f25e903cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:58:54.788648 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f5e8df-4f3e-429d-b4b7-857f25e903cb None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:58:54.788925 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f5e8df-4f3e-429d-b4b7-857f25e903cb None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:58:54.789189 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0f5e8df-4f3e-429d-b4b7-857f25e903cb None admin] PATH_INFO: `/v3/users/2c1dc01ea50848449386ae75b0c5ece6` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:58:54.790437 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f5e8df-4f3e-429d-b4b7-857f25e903cb None admin] RBAC: Authorizing `identity:get_user(user_id=2c1dc01ea50848449386ae75b0c5ece6)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:58:54.792000 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0f5e8df-4f3e-429d-b4b7-857f25e903cb None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:58:54.793647 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 158/308] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Jul 13 21:58:54 2022] GET /identity/v3/users/2c1dc01ea50848449386ae75b0c5ece6 => generated 293 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 21:58:54.794419 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f5e8df-4f3e-429d-b4b7-857f25e903cb None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:58:54.814379 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3385373-1606-4349-bba5-5fbfcc19d481 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:58:54.815216 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3385373-1606-4349-bba5-5fbfcc19d481 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:58:54.815507 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3385373-1606-4349-bba5-5fbfcc19d481 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:58:54.815760 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3385373-1606-4349-bba5-5fbfcc19d481 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:58:54.816917 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3385373-1606-4349-bba5-5fbfcc19d481 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:58:54.818381 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3385373-1606-4349-bba5-5fbfcc19d481 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:58:54.820025 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 151/309] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Jul 13 21:58:54 2022] GET /identity/v3/domains/default => generated 199 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 21:58:54.820756 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3385373-1606-4349-bba5-5fbfcc19d481 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:58:54.897321 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0f5e8df-4f3e-429d-b4b7-857f25e903cb None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:58:54.902398 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:58:54.918139 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3385373-1606-4349-bba5-5fbfcc19d481 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:58:54.929879 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e41ac3a-fae5-458b-a88a-f7d84e5b31c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:58:54.930785 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e41ac3a-fae5-458b-a88a-f7d84e5b31c3 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:58:54.931061 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e41ac3a-fae5-458b-a88a-f7d84e5b31c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:58:54.931316 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e41ac3a-fae5-458b-a88a-f7d84e5b31c3 None admin] PATH_INFO: `/v3/users/286300b3ff1541089d0295569ac25679` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:58:54.932551 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e41ac3a-fae5-458b-a88a-f7d84e5b31c3 None admin] RBAC: Authorizing `identity:get_user(user_id=286300b3ff1541089d0295569ac25679)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:58:54.933995 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e41ac3a-fae5-458b-a88a-f7d84e5b31c3 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:58:54.935707 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 159/310] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Jul 13 21:58:54 2022] GET /identity/v3/users/286300b3ff1541089d0295569ac25679 => generated 297 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 21:58:54.936434 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e41ac3a-fae5-458b-a88a-f7d84e5b31c3 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:58:54.955552 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b75034-1322-4d31-9666-8658cedd487d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:58:54.956227 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b75034-1322-4d31-9666-8658cedd487d None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:58:54.956506 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b75034-1322-4d31-9666-8658cedd487d None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:58:54.956775 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99b75034-1322-4d31-9666-8658cedd487d None admin] PATH_INFO: `/v3/roles` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:58:54.957533 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b75034-1322-4d31-9666-8658cedd487d None admin] RBAC: Adding query filter params (name=reader) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 21:58:54.957828 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b75034-1322-4d31-9666-8658cedd487d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:58:54.959249 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99b75034-1322-4d31-9666-8658cedd487d None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:58:54.967665 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 152/311] 10.222.0.52 () {56 vars in 1101 bytes} [Wed Jul 13 21:58:54 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 21:58:54.968575 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b75034-1322-4d31-9666-8658cedd487d None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:58:54.976683 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:58:55.041555 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e41ac3a-fae5-458b-a88a-f7d84e5b31c3 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:58:55.066403 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99b75034-1322-4d31-9666-8658cedd487d None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:58:55.075364 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ba026a-6aee-48bf-80e9-542da3cb5d7e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:58:55.076239 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ba026a-6aee-48bf-80e9-542da3cb5d7e None admin] REQUEST_METHOD: `PUT` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:58:55.076517 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ba026a-6aee-48bf-80e9-542da3cb5d7e None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:58:55.076792 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ba026a-6aee-48bf-80e9-542da3cb5d7e None admin] PATH_INFO: `/v3/system/users/2c1dc01ea50848449386ae75b0c5ece6/roles/2a5e86976394498abb6310697cdb3f8b` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:58:55.078286 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ba026a-6aee-48bf-80e9-542da3cb5d7e None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=2c1dc01ea50848449386ae75b0c5ece6, role_id=2a5e86976394498abb6310697cdb3f8b)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:58:55.079732 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ba026a-6aee-48bf-80e9-542da3cb5d7e None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:58:55.089699 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 160/312] 10.222.0.52 () {58 vars in 1334 bytes} [Wed Jul 13 21:58:54 2022] PUT /identity/v3/system/users/2c1dc01ea50848449386ae75b0c5ece6/roles/2a5e86976394498abb6310697cdb3f8b => generated 0 bytes in 153 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 21:58:55.106666 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ba026a-6aee-48bf-80e9-542da3cb5d7e None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:58:55.106802 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2cb1eb-0026-4c14-ad63-06336e4de7cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:58:55.106802 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2cb1eb-0026-4c14-ad63-06336e4de7cf None admin] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:58:55.106802 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2cb1eb-0026-4c14-ad63-06336e4de7cf None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:58:55.106802 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d2cb1eb-0026-4c14-ad63-06336e4de7cf None admin] PATH_INFO: `/v3/groups` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:58:55.107076 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2cb1eb-0026-4c14-ad63-06336e4de7cf None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:58:55.107076 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d2cb1eb-0026-4c14-ad63-06336e4de7cf None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:58:55.120892 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 153/313] 10.222.0.52 () {60 vars in 1134 bytes} [Wed Jul 13 21:58:54 2022] POST /identity/v3/groups => generated 229 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 21:58:55.121874 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2cb1eb-0026-4c14-ad63-06336e4de7cf None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:58:55.143211 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:58:55.232379 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:58:55.235782 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ba026a-6aee-48bf-80e9-542da3cb5d7e None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:58:55.281827 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4b945b-2fc3-427c-8b08-42453d36828d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:58:55.282688 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4b945b-2fc3-427c-8b08-42453d36828d None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:58:55.282969 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4b945b-2fc3-427c-8b08-42453d36828d None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:58:55.283265 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b4b945b-2fc3-427c-8b08-42453d36828d None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:58:55.292538 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d2cb1eb-0026-4c14-ad63-06336e4de7cf None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:58:55.335519 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db0f280-e09e-4b47-b3d8-4a45127f438b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:58:55.336419 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db0f280-e09e-4b47-b3d8-4a45127f438b None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:58:55.336698 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db0f280-e09e-4b47-b3d8-4a45127f438b None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:58:55.336970 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4db0f280-e09e-4b47-b3d8-4a45127f438b None admin] PATH_INFO: `/v3/users/54013c244a4e4e46b8ae8d0d714138bf` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:58:55.353045 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db0f280-e09e-4b47-b3d8-4a45127f438b None admin] RBAC: Authorizing `identity:get_user(user_id=54013c244a4e4e46b8ae8d0d714138bf)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:58:55.354765 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4db0f280-e09e-4b47-b3d8-4a45127f438b None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:58:55.356705 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 154/314] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Jul 13 21:58:55 2022] GET /identity/v3/users/54013c244a4e4e46b8ae8d0d714138bf => generated 293 bytes in 235 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 21:58:55.360998 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4b945b-2fc3-427c-8b08-42453d36828d None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:58:55.362349 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db0f280-e09e-4b47-b3d8-4a45127f438b None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:58:55.362589 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b4b945b-2fc3-427c-8b08-42453d36828d None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:58:55.459108 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:58:55.473408 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:58:55.482415 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 161/315] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Jul 13 21:58:55 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 13 21:58:55.484948 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4b945b-2fc3-427c-8b08-42453d36828d None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:58:55.513837 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4db0f280-e09e-4b47-b3d8-4a45127f438b None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:58:55.557348 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7646566-9423-4c1d-b573-15d6bb73d3b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:58:55.559077 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7646566-9423-4c1d-b573-15d6bb73d3b6 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:58:55.559814 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7646566-9423-4c1d-b573-15d6bb73d3b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:58:55.561159 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7646566-9423-4c1d-b573-15d6bb73d3b6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:58:55.563119 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7646566-9423-4c1d-b573-15d6bb73d3b6 None admin] RBAC: Adding query filter params (role.id=abb8511cb7d54b249e85bbc7507f0317, user.id=54013c244a4e4e46b8ae8d0d714138bf, scope.system=all) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 21:58:55.563633 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7646566-9423-4c1d-b573-15d6bb73d3b6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:58:55.565647 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7646566-9423-4c1d-b573-15d6bb73d3b6 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:58:55.579084 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 155/316] 10.222.0.52 () {56 vars in 1308 bytes} [Wed Jul 13 21:58:55 2022] GET /identity/v3/role_assignments?user.id=54013c244a4e4e46b8ae8d0d714138bf&scope.system=all&role.id=abb8511cb7d54b249e85bbc7507f0317 => generated 226 bytes in 211 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 21:58:55.606669 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b4b945b-2fc3-427c-8b08-42453d36828d None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:58:55.644450 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a89673-ed0a-4b5c-8725-c7161b33ce02 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:58:55.645362 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a89673-ed0a-4b5c-8725-c7161b33ce02 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:58:55.645637 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a89673-ed0a-4b5c-8725-c7161b33ce02 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:58:55.645901 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a89673-ed0a-4b5c-8725-c7161b33ce02 None admin] PATH_INFO: `/v3/projects/4896b13c732d49d09947c2baaf9dc04d` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:58:55.647174 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a89673-ed0a-4b5c-8725-c7161b33ce02 None admin] RBAC: Authorizing `identity:get_project(project_id=4896b13c732d49d09947c2baaf9dc04d)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:58:55.648635 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48a89673-ed0a-4b5c-8725-c7161b33ce02 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:58:55.651127 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 162/317] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Jul 13 21:58:55 2022] GET /identity/v3/projects/4896b13c732d49d09947c2baaf9dc04d => generated 302 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 21:58:55.659688 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7646566-9423-4c1d-b573-15d6bb73d3b6 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:58:55.769758 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7646566-9423-4c1d-b573-15d6bb73d3b6 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:58:55.798803 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:58:55.807092 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-812fe17f-b7ed-4599-9667-9ae1ed34eda2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:58:55.808055 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812fe17f-b7ed-4599-9667-9ae1ed34eda2 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:58:55.808528 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812fe17f-b7ed-4599-9667-9ae1ed34eda2 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:58:55.809005 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-812fe17f-b7ed-4599-9667-9ae1ed34eda2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:58:55.810037 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812fe17f-b7ed-4599-9667-9ae1ed34eda2 None admin] RBAC: Adding query filter params (role.id=abb8511cb7d54b249e85bbc7507f0317, scope.project.id=4896b13c732d49d09947c2baaf9dc04d, user.id=286300b3ff1541089d0295569ac25679) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 21:58:55.810576 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812fe17f-b7ed-4599-9667-9ae1ed34eda2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:58:55.812217 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-812fe17f-b7ed-4599-9667-9ae1ed34eda2 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:58:55.822600 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 156/318] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Jul 13 21:58:55 2022] GET /identity/v3/role_assignments?user.id=286300b3ff1541089d0295569ac25679&scope.project.id=4896b13c732d49d09947c2baaf9dc04d&role.id=abb8511cb7d54b249e85bbc7507f0317 => generated 608 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 21:58:57.174431 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf055a7-5539-4358-9cd8-3f195cefab66 None None] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:58:57.175015 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf055a7-5539-4358-9cd8-3f195cefab66 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:58:57.175015 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5bf055a7-5539-4358-9cd8-3f195cefab66 None None] PATH_INFO: `/` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:58:57.176568 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 163/319] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jul 13 21:58:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 21:58:57.181506 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334063ac-d0f0-439b-b07a-3d697b6cd582 None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:58:57.182631 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334063ac-d0f0-439b-b07a-3d697b6cd582 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:58:57.182631 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-334063ac-d0f0-439b-b07a-3d697b6cd582 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:58:57.198927 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-334063ac-d0f0-439b-b07a-3d697b6cd582 None None] MFA Rules not processed for user `cc99c93da5074cf5a48bc2aa9a92c5e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:58:57.200445 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-334063ac-d0f0-439b-b07a-3d697b6cd582 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:58:57.242130 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b611f45-b2d6-4004-ae87-0bae0e7bf608 None None] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:58:57.242365 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b611f45-b2d6-4004-ae87-0bae0e7bf608 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:58:57.242687 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b611f45-b2d6-4004-ae87-0bae0e7bf608 None None] PATH_INFO: `/` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:58:57.244260 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 164/320] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jul 13 21:58:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 13 21:58:57.249259 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75156704-40ba-4230-b028-3d0e53e5852b None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:58:57.249520 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75156704-40ba-4230-b028-3d0e53e5852b None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:58:57.249802 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75156704-40ba-4230-b028-3d0e53e5852b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:58:57.266683 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-75156704-40ba-4230-b028-3d0e53e5852b None None] MFA Rules not processed for user `cc99c93da5074cf5a48bc2aa9a92c5e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:58:57.267901 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-75156704-40ba-4230-b028-3d0e53e5852b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:58:57.272348 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 157/321] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jul 13 21:58:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 13 21:58:57.296684 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334063ac-d0f0-439b-b07a-3d697b6cd582 None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:58:57.332914 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:58:57.350513 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 165/322] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jul 13 21:58:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 13 21:58:57.362238 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75156704-40ba-4230-b028-3d0e53e5852b None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:58:57.402643 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-334063ac-d0f0-439b-b07a-3d697b6cd582 None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:58:57.426624 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:58:57.439035 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5212e75-186a-4589-8686-1fe026ad4cad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:58:57.439869 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5212e75-186a-4589-8686-1fe026ad4cad None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:58:57.440149 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5212e75-186a-4589-8686-1fe026ad4cad None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:58:57.440417 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5212e75-186a-4589-8686-1fe026ad4cad None admin] PATH_INFO: `/v3/roles/member` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:58:57.451634 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5212e75-186a-4589-8686-1fe026ad4cad None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 21:58:57.459508 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application [None req-a5212e75-186a-4589-8686-1fe026ad4cad None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 13 21:58:57.459508 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 21:58:57.459508 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 21:58:57.459508 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 21:58:57.459508 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 21:58:57.459508 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 21:58:57.459508 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 21:58:57.459508 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 21:58:57.459508 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 21:58:57.459508 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 21:58:57.459508 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 21:58:57.459508 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 21:58:57.459508 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 21:58:57.459508 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 21:58:57.459508 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 21:58:57.459508 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 21:58:57.460475 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 21:58:57.460475 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 21:58:57.460475 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 21:58:57.460475 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 21:58:57.460475 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 21:58:57.460475 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 21:58:57.460475 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 21:58:57.460475 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 21:58:57.460475 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 21:58:57.460475 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 21:58:57.460475 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 21:58:57.460475 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application with Lock( Jul 13 21:58:57.460475 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 21:58:57.460475 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._enter() Jul 13 21:58:57.460475 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 21:58:57.460475 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 21:58:57.461312 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 21:58:57.461312 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.creator() Jul 13 21:58:57.461312 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 21:58:57.461312 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application created_value = creator( Jul 13 21:58:57.461312 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 21:58:57.461312 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 21:58:57.461312 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 21:58:57.461312 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 21:58:57.461312 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 21:58:57.461312 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 21:58:57.461312 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 13 21:58:57.461312 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Jul 13 21:58:57.461312 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 158/323] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Jul 13 21:58:57 2022] GET /identity/v3/roles/member => generated 84 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 21:58:57.467083 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5212e75-186a-4589-8686-1fe026ad4cad None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:58:57.470081 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75156704-40ba-4230-b028-3d0e53e5852b None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:58:57.505651 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a74286-5d82-46f2-805b-6f8e96b75114 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:58:57.506467 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a74286-5d82-46f2-805b-6f8e96b75114 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:58:57.506777 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a74286-5d82-46f2-805b-6f8e96b75114 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:58:57.507052 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a74286-5d82-46f2-805b-6f8e96b75114 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:58:57.508219 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a74286-5d82-46f2-805b-6f8e96b75114 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:58:57.509687 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a74286-5d82-46f2-805b-6f8e96b75114 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:58:57.511558 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 166/324] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Jul 13 21:58:57 2022] GET /identity/v3/domains/default => generated 199 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 21:58:57.516786 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a74286-5d82-46f2-805b-6f8e96b75114 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:58:57.577036 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5212e75-186a-4589-8686-1fe026ad4cad None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:58:57.589452 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:58:57.615936 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0eba2d1c-19aa-4f42-93dc-794a0d43e6b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:58:57.616773 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eba2d1c-19aa-4f42-93dc-794a0d43e6b5 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:58:57.617052 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eba2d1c-19aa-4f42-93dc-794a0d43e6b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:58:57.617329 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0eba2d1c-19aa-4f42-93dc-794a0d43e6b5 None admin] PATH_INFO: `/v3/roles` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:58:57.618088 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eba2d1c-19aa-4f42-93dc-794a0d43e6b5 None admin] RBAC: Adding query filter params (name=member) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 21:58:57.618393 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eba2d1c-19aa-4f42-93dc-794a0d43e6b5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:58:57.619917 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0eba2d1c-19aa-4f42-93dc-794a0d43e6b5 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:58:57.625706 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a74286-5d82-46f2-805b-6f8e96b75114 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:58:57.628003 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 159/325] 10.222.0.52 () {56 vars in 1101 bytes} [Wed Jul 13 21:58:57 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 21:58:57.628003 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5caf9da3-a78d-4385-a4ec-bc4677ca8f0b None None] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:58:57.628003 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5caf9da3-a78d-4385-a4ec-bc4677ca8f0b None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:58:57.628003 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5caf9da3-a78d-4385-a4ec-bc4677ca8f0b None None] PATH_INFO: `/` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:58:57.631164 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 160/326] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jul 13 21:58:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 21:58:57.633494 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5caf9da3-a78d-4385-a4ec-bc4677ca8f0b None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:58:57.661500 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef73082a-4a82-49ca-a692-b9282108e467 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:58:57.662176 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef73082a-4a82-49ca-a692-b9282108e467 None admin] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:58:57.662457 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef73082a-4a82-49ca-a692-b9282108e467 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:58:57.662757 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef73082a-4a82-49ca-a692-b9282108e467 None admin] PATH_INFO: `/v3/groups` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:58:57.663450 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef73082a-4a82-49ca-a692-b9282108e467 None admin] RBAC: Authorizing `identity:create_group()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:58:57.664909 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef73082a-4a82-49ca-a692-b9282108e467 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:58:57.686326 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 167/327] 10.222.0.52 () {60 vars in 1134 bytes} [Wed Jul 13 21:58:57 2022] POST /identity/v3/groups => generated 226 bytes in 164 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 13 21:58:57.686326 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b2560a-c64f-4c4d-b609-2653f2f498d1 None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:58:57.686326 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b2560a-c64f-4c4d-b609-2653f2f498d1 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:58:57.686326 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6b2560a-c64f-4c4d-b609-2653f2f498d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:58:57.691697 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:58:57.708516 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-c6b2560a-c64f-4c4d-b609-2653f2f498d1 None None] MFA Rules not processed for user `cc99c93da5074cf5a48bc2aa9a92c5e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:58:57.709754 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-c6b2560a-c64f-4c4d-b609-2653f2f498d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:58:57.758248 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5caf9da3-a78d-4385-a4ec-bc4677ca8f0b None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:58:57.790176 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 168/328] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jul 13 21:58:57 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 13 21:58:57.802464 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e44fdc-f1b2-4741-aa9e-e1c582b89024 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:58:57.803452 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e44fdc-f1b2-4741-aa9e-e1c582b89024 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:58:57.803735 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e44fdc-f1b2-4741-aa9e-e1c582b89024 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:58:57.804016 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0e44fdc-f1b2-4741-aa9e-e1c582b89024 None admin] PATH_INFO: `/v3/users/2c1dc01ea50848449386ae75b0c5ece6` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:58:57.805296 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e44fdc-f1b2-4741-aa9e-e1c582b89024 None admin] RBAC: Authorizing `identity:get_user(user_id=2c1dc01ea50848449386ae75b0c5ece6)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:58:57.806899 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0e44fdc-f1b2-4741-aa9e-e1c582b89024 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:58:57.809556 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 161/329] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Jul 13 21:58:57 2022] GET /identity/v3/users/2c1dc01ea50848449386ae75b0c5ece6 => generated 293 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 21:58:57.813378 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b2560a-c64f-4c4d-b609-2653f2f498d1 None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:58:57.815065 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e44fdc-f1b2-4741-aa9e-e1c582b89024 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:58:57.880471 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:58:57.933237 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0e44fdc-f1b2-4741-aa9e-e1c582b89024 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:58:57.944592 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6b2560a-c64f-4c4d-b609-2653f2f498d1 None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:58:57.947162 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:58:57.984033 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09ac2ca-eb7d-42de-856b-93f38a94b43e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:58:57.984716 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09ac2ca-eb7d-42de-856b-93f38a94b43e None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:58:57.984998 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09ac2ca-eb7d-42de-856b-93f38a94b43e None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:58:57.985300 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b09ac2ca-eb7d-42de-856b-93f38a94b43e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:58:57.986129 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b09ac2ca-eb7d-42de-856b-93f38a94b43e None admin] RBAC: Adding query filter params (role.id=2a5e86976394498abb6310697cdb3f8b, user.id=2c1dc01ea50848449386ae75b0c5ece6, scope.system=all) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 21:58:57.986439 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b09ac2ca-eb7d-42de-856b-93f38a94b43e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:58:57.987919 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b09ac2ca-eb7d-42de-856b-93f38a94b43e None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:58:57.992511 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba96e886-5c43-4720-b69b-fa83bae69238 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:58:57.993521 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba96e886-5c43-4720-b69b-fa83bae69238 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:58:57.993982 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba96e886-5c43-4720-b69b-fa83bae69238 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:58:57.994344 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba96e886-5c43-4720-b69b-fa83bae69238 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:58:58.000950 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 162/330] 10.222.0.52 () {56 vars in 1308 bytes} [Wed Jul 13 21:58:57 2022] GET /identity/v3/role_assignments?user.id=2c1dc01ea50848449386ae75b0c5ece6&scope.system=all&role.id=2a5e86976394498abb6310697cdb3f8b => generated 510 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 21:58:58.008827 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba96e886-5c43-4720-b69b-fa83bae69238 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 21:58:58.018009 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application [None req-ba96e886-5c43-4720-b69b-fa83bae69238 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 13 21:58:58.018009 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 21:58:58.018009 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 21:58:58.018009 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 21:58:58.018009 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 21:58:58.018009 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 21:58:58.018009 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 21:58:58.018009 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 21:58:58.018009 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 21:58:58.018009 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 21:58:58.018009 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 21:58:58.018009 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 21:58:58.018009 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 21:58:58.018009 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 21:58:58.018009 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 21:58:58.018009 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 21:58:58.019033 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 21:58:58.019033 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 21:58:58.019033 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 21:58:58.019033 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 21:58:58.019033 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 21:58:58.019033 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 21:58:58.019033 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 21:58:58.019033 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 21:58:58.019033 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 21:58:58.019033 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 21:58:58.019033 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 21:58:58.019033 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application with Lock( Jul 13 21:58:58.019033 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 21:58:58.019033 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._enter() Jul 13 21:58:58.019033 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 21:58:58.019033 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 21:58:58.019816 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 21:58:58.019816 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.creator() Jul 13 21:58:58.019816 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 21:58:58.019816 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application created_value = creator( Jul 13 21:58:58.019816 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 21:58:58.019816 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 21:58:58.019816 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 21:58:58.019816 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 21:58:58.019816 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 21:58:58.019816 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 21:58:58.019816 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 13 21:58:58.019816 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Jul 13 21:58:58.019816 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 169/331] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Jul 13 21:58:57 2022] GET /identity/v3/roles/reader => generated 84 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 21:58:58.026105 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09ac2ca-eb7d-42de-856b-93f38a94b43e None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:58:58.150331 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b09ac2ca-eb7d-42de-856b-93f38a94b43e None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:58:58.191609 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0583d150-c02c-4426-94e2-71bb64387d6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:58:58.192384 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0583d150-c02c-4426-94e2-71bb64387d6b None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:58:58.192568 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0583d150-c02c-4426-94e2-71bb64387d6b None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:58:58.192844 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0583d150-c02c-4426-94e2-71bb64387d6b None admin] PATH_INFO: `/v3/roles` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:58:58.193670 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0583d150-c02c-4426-94e2-71bb64387d6b None admin] RBAC: Adding query filter params (name=reader) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 21:58:58.193973 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0583d150-c02c-4426-94e2-71bb64387d6b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:58:58.195535 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0583d150-c02c-4426-94e2-71bb64387d6b None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:58:58.206833 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 163/332] 10.222.0.52 () {56 vars in 1101 bytes} [Wed Jul 13 21:58:58 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 21:58:58.214480 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba96e886-5c43-4720-b69b-fa83bae69238 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:58:58.325911 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba96e886-5c43-4720-b69b-fa83bae69238 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:58:58.343753 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:58:58.369580 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ffbbf73-5b6a-4117-9b09-a63c47744230 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:58:58.370358 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffbbf73-5b6a-4117-9b09-a63c47744230 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:58:58.370716 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffbbf73-5b6a-4117-9b09-a63c47744230 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:58:58.370992 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ffbbf73-5b6a-4117-9b09-a63c47744230 None admin] PATH_INFO: `/v3/users/54013c244a4e4e46b8ae8d0d714138bf` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:58:58.372225 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ffbbf73-5b6a-4117-9b09-a63c47744230 None admin] RBAC: Authorizing `identity:get_user(user_id=54013c244a4e4e46b8ae8d0d714138bf)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:58:58.373697 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ffbbf73-5b6a-4117-9b09-a63c47744230 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:58:58.375738 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 170/333] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Jul 13 21:58:58 2022] GET /identity/v3/users/54013c244a4e4e46b8ae8d0d714138bf => generated 293 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 21:58:58.381408 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0583d150-c02c-4426-94e2-71bb64387d6b None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:58:58.414912 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:58:58.486326 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0583d150-c02c-4426-94e2-71bb64387d6b None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:58:58.524085 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92b302c2-d93e-4836-a7d4-cf230e97dcc2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:58:58.525293 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b302c2-d93e-4836-a7d4-cf230e97dcc2 None admin] REQUEST_METHOD: `PUT` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:58:58.525913 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b302c2-d93e-4836-a7d4-cf230e97dcc2 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:58:58.526363 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92b302c2-d93e-4836-a7d4-cf230e97dcc2 None admin] PATH_INFO: `/v3/system/users/54013c244a4e4e46b8ae8d0d714138bf/roles/abb8511cb7d54b249e85bbc7507f0317` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:58:58.528070 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b302c2-d93e-4836-a7d4-cf230e97dcc2 None admin] RBAC: Authorizing `identity:create_system_grant_for_user(user_id=54013c244a4e4e46b8ae8d0d714138bf, role_id=abb8511cb7d54b249e85bbc7507f0317)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:58:58.529635 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92b302c2-d93e-4836-a7d4-cf230e97dcc2 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:58:58.544155 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 164/334] 10.222.0.52 () {58 vars in 1334 bytes} [Wed Jul 13 21:58:58 2022] PUT /identity/v3/system/users/54013c244a4e4e46b8ae8d0d714138bf/roles/abb8511cb7d54b249e85bbc7507f0317 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 13 21:59:00.256097 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4dca92-5a55-4cf6-91ad-00a04e9c99f1 None None] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:00.256616 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4dca92-5a55-4cf6-91ad-00a04e9c99f1 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:00.256690 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d4dca92-5a55-4cf6-91ad-00a04e9c99f1 None None] PATH_INFO: `/` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:00.258319 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 171/335] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jul 13 21:59:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 21:59:00.274666 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f98166-2f11-4756-aa7d-91fb76c1edbd None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:00.275803 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f98166-2f11-4756-aa7d-91fb76c1edbd None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:00.276442 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51f98166-2f11-4756-aa7d-91fb76c1edbd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:00.284535 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9c9188-8721-4a47-b9c0-29661bf49537 None None] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:00.284815 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9c9188-8721-4a47-b9c0-29661bf49537 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:00.285105 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a9c9188-8721-4a47-b9c0-29661bf49537 None None] PATH_INFO: `/` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:00.288127 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 172/336] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jul 13 21:59:00 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 21:59:00.295158 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c677618c-590d-4db2-94da-b68373a764fa None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:00.295453 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c677618c-590d-4db2-94da-b68373a764fa None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:00.295729 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c677618c-590d-4db2-94da-b68373a764fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:00.303587 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-51f98166-2f11-4756-aa7d-91fb76c1edbd None None] MFA Rules not processed for user `cc99c93da5074cf5a48bc2aa9a92c5e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:59:00.305153 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-51f98166-2f11-4756-aa7d-91fb76c1edbd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:59:00.315847 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-c677618c-590d-4db2-94da-b68373a764fa None None] MFA Rules not processed for user `cc99c93da5074cf5a48bc2aa9a92c5e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:59:00.317042 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-c677618c-590d-4db2-94da-b68373a764fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:59:00.400210 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 173/337] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jul 13 21:59:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 13 21:59:00.423268 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c677618c-590d-4db2-94da-b68373a764fa None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:00.434475 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 165/338] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jul 13 21:59:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 13 21:59:00.554809 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c677618c-590d-4db2-94da-b68373a764fa None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:00.560078 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f98166-2f11-4756-aa7d-91fb76c1edbd None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:00.571544 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:59:00.595856 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc14f4c5-b37c-4596-8994-16bc06565b6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:00.596813 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc14f4c5-b37c-4596-8994-16bc06565b6d None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:00.597108 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc14f4c5-b37c-4596-8994-16bc06565b6d None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:00.597383 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc14f4c5-b37c-4596-8994-16bc06565b6d None admin] PATH_INFO: `/v3/roles/member` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:00.609520 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc14f4c5-b37c-4596-8994-16bc06565b6d None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 21:59:00.618132 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application [None req-dc14f4c5-b37c-4596-8994-16bc06565b6d None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 13 21:59:00.618132 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 21:59:00.618132 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 21:59:00.618132 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 21:59:00.618132 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 21:59:00.618132 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 21:59:00.618132 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 21:59:00.618132 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 21:59:00.618132 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 21:59:00.618132 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 21:59:00.618132 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 21:59:00.618132 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 21:59:00.618132 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 21:59:00.618132 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 21:59:00.618132 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 21:59:00.618132 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 21:59:00.619042 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 21:59:00.619042 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 21:59:00.619042 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 21:59:00.619042 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 21:59:00.619042 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 21:59:00.619042 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 21:59:00.619042 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 21:59:00.619042 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 21:59:00.619042 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 21:59:00.619042 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 21:59:00.619042 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 21:59:00.619042 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application with Lock( Jul 13 21:59:00.619042 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 21:59:00.619042 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._enter() Jul 13 21:59:00.619042 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 21:59:00.619042 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 21:59:00.619895 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 21:59:00.619895 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.creator() Jul 13 21:59:00.619895 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 21:59:00.619895 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application created_value = creator( Jul 13 21:59:00.619895 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 21:59:00.619895 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 21:59:00.619895 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 21:59:00.619895 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 21:59:00.619895 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 21:59:00.619895 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 21:59:00.619895 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 13 21:59:00.619895 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Jul 13 21:59:00.624466 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 174/339] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Jul 13 21:59:00 2022] GET /identity/v3/roles/member => generated 84 bytes in 197 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 21:59:00.625957 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc14f4c5-b37c-4596-8994-16bc06565b6d None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:00.640351 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:59:00.741134 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc14f4c5-b37c-4596-8994-16bc06565b6d None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:00.777728 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51f98166-2f11-4756-aa7d-91fb76c1edbd None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:00.787256 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f28273da-4cf2-4c79-afab-76b83a593b2d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:00.787944 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28273da-4cf2-4c79-afab-76b83a593b2d None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:00.788222 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28273da-4cf2-4c79-afab-76b83a593b2d None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:00.788493 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f28273da-4cf2-4c79-afab-76b83a593b2d None admin] PATH_INFO: `/v3/roles` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:00.789261 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f28273da-4cf2-4c79-afab-76b83a593b2d None admin] RBAC: Adding query filter params (name=member) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 21:59:00.789564 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f28273da-4cf2-4c79-afab-76b83a593b2d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:00.792557 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f28273da-4cf2-4c79-afab-76b83a593b2d None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:00.801851 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 175/340] 10.222.0.52 () {56 vars in 1101 bytes} [Wed Jul 13 21:59:00 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 21:59:00.803359 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ec2cd3-131e-4636-855b-e90e8757f25e None None] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:00.803786 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ec2cd3-131e-4636-855b-e90e8757f25e None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:00.805285 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9ec2cd3-131e-4636-855b-e90e8757f25e None None] PATH_INFO: `/` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:00.806895 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 176/341] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jul 13 21:59:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 21:59:00.813094 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beffdfb1-064a-4b0f-b9f3-00a96970a68f None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:00.813637 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beffdfb1-064a-4b0f-b9f3-00a96970a68f None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:00.814110 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beffdfb1-064a-4b0f-b9f3-00a96970a68f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:00.823518 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66904e6-8ece-4249-a319-a6f92d88d740 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:00.824690 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66904e6-8ece-4249-a319-a6f92d88d740 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:00.824965 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66904e6-8ece-4249-a319-a6f92d88d740 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:00.825235 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d66904e6-8ece-4249-a319-a6f92d88d740 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:00.834097 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-beffdfb1-064a-4b0f-b9f3-00a96970a68f None None] MFA Rules not processed for user `cc99c93da5074cf5a48bc2aa9a92c5e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:59:00.835618 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-beffdfb1-064a-4b0f-b9f3-00a96970a68f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:59:00.836782 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d66904e6-8ece-4249-a319-a6f92d88d740 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 21:59:00.847358 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application [None req-d66904e6-8ece-4249-a319-a6f92d88d740 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 13 21:59:00.847358 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 21:59:00.847358 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 21:59:00.847358 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 21:59:00.847358 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 21:59:00.847358 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 21:59:00.847358 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 21:59:00.847358 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 21:59:00.847358 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 21:59:00.847358 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 21:59:00.847358 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 21:59:00.847358 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 21:59:00.847358 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 21:59:00.847358 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 21:59:00.847358 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 21:59:00.847358 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 21:59:00.848241 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 21:59:00.848241 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 21:59:00.848241 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 21:59:00.848241 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 21:59:00.848241 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 21:59:00.848241 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 21:59:00.848241 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 21:59:00.848241 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 21:59:00.848241 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 21:59:00.848241 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 21:59:00.848241 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 21:59:00.848241 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application with Lock( Jul 13 21:59:00.848241 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 21:59:00.848241 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._enter() Jul 13 21:59:00.848241 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 21:59:00.848241 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 21:59:00.849062 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 21:59:00.849062 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.creator() Jul 13 21:59:00.849062 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 21:59:00.849062 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application created_value = creator( Jul 13 21:59:00.849062 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 21:59:00.849062 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 21:59:00.849062 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 21:59:00.849062 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 21:59:00.849062 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 21:59:00.849062 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 21:59:00.849062 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 13 21:59:00.849062 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Jul 13 21:59:00.849062 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 166/342] 10.222.0.52 () {56 vars in 1114 bytes} [Wed Jul 13 21:59:00 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 289 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 13 21:59:00.854971 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66904e6-8ece-4249-a319-a6f92d88d740 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:00.877589 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:59:00.919964 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 177/343] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jul 13 21:59:00 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 13 21:59:00.921134 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beffdfb1-064a-4b0f-b9f3-00a96970a68f None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:00.965085 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:59:00.978231 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d66904e6-8ece-4249-a319-a6f92d88d740 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:01.022653 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5afbaff3-cf97-41bd-84b2-80a2a9f5e7f8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:01.024664 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afbaff3-cf97-41bd-84b2-80a2a9f5e7f8 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:01.025518 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afbaff3-cf97-41bd-84b2-80a2a9f5e7f8 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:01.026030 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5afbaff3-cf97-41bd-84b2-80a2a9f5e7f8 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:01.063359 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beffdfb1-064a-4b0f-b9f3-00a96970a68f None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:01.106743 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07289213-9a96-4014-9e2a-19a1a3407e35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:01.107698 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07289213-9a96-4014-9e2a-19a1a3407e35 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:01.108134 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07289213-9a96-4014-9e2a-19a1a3407e35 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:01.108601 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07289213-9a96-4014-9e2a-19a1a3407e35 None admin] PATH_INFO: `/v3/roles` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:01.109539 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07289213-9a96-4014-9e2a-19a1a3407e35 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 21:59:01.109995 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07289213-9a96-4014-9e2a-19a1a3407e35 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:01.111574 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07289213-9a96-4014-9e2a-19a1a3407e35 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:01.112592 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5afbaff3-cf97-41bd-84b2-80a2a9f5e7f8 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:01.116128 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5afbaff3-cf97-41bd-84b2-80a2a9f5e7f8 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:01.130865 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 178/344] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Jul 13 21:59:00 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 21:59:01.130865 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07289213-9a96-4014-9e2a-19a1a3407e35 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:01.170340 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:59:01.261428 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07289213-9a96-4014-9e2a-19a1a3407e35 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:01.302994 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845d64ad-f0a0-4dc0-b164-7e1255eaf0ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:01.304584 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845d64ad-f0a0-4dc0-b164-7e1255eaf0ca None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:01.305012 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845d64ad-f0a0-4dc0-b164-7e1255eaf0ca None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:01.305427 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-845d64ad-f0a0-4dc0-b164-7e1255eaf0ca None admin] PATH_INFO: `/v3/roles/member` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:01.313806 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:59:01.319907 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-845d64ad-f0a0-4dc0-b164-7e1255eaf0ca None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 21:59:01.331352 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application [None req-845d64ad-f0a0-4dc0-b164-7e1255eaf0ca None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 13 21:59:01.331352 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 21:59:01.331352 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 21:59:01.331352 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 21:59:01.331352 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 21:59:01.331352 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 21:59:01.331352 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 21:59:01.331352 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 21:59:01.331352 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 21:59:01.331352 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 21:59:01.331352 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 21:59:01.331352 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 21:59:01.331352 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 21:59:01.331352 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 21:59:01.331352 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 21:59:01.331352 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 21:59:01.332444 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 21:59:01.332444 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 21:59:01.332444 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 21:59:01.332444 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 21:59:01.332444 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 21:59:01.332444 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 21:59:01.332444 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 21:59:01.332444 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 21:59:01.332444 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 21:59:01.332444 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 21:59:01.332444 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 21:59:01.332444 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application with Lock( Jul 13 21:59:01.332444 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 21:59:01.332444 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._enter() Jul 13 21:59:01.332444 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 21:59:01.332444 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 21:59:01.333263 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 21:59:01.333263 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.creator() Jul 13 21:59:01.333263 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 21:59:01.333263 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application created_value = creator( Jul 13 21:59:01.333263 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 21:59:01.333263 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 21:59:01.333263 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 21:59:01.333263 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 21:59:01.333263 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 21:59:01.333263 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 21:59:01.333263 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 13 21:59:01.333263 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Jul 13 21:59:01.335723 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 179/345] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Jul 13 21:59:01 2022] GET /identity/v3/roles/member => generated 84 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 21:59:01.337227 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845d64ad-f0a0-4dc0-b164-7e1255eaf0ca None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:01.345269 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 167/346] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Jul 13 21:59:00 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 496 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 13 21:59:01.348712 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5afbaff3-cf97-41bd-84b2-80a2a9f5e7f8 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:01.459272 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-845d64ad-f0a0-4dc0-b164-7e1255eaf0ca None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:01.469370 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5afbaff3-cf97-41bd-84b2-80a2a9f5e7f8 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:01.486083 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:59:01.498238 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0563b4-15a4-4733-a2ad-e5a9564ba472 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:01.499144 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0563b4-15a4-4733-a2ad-e5a9564ba472 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:01.499705 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0563b4-15a4-4733-a2ad-e5a9564ba472 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:01.500268 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b0563b4-15a4-4733-a2ad-e5a9564ba472 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:01.519232 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330d4767-d08b-42b4-baec-49233b57793a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:01.519937 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330d4767-d08b-42b4-baec-49233b57793a None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:01.520219 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330d4767-d08b-42b4-baec-49233b57793a None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:01.520497 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-330d4767-d08b-42b4-baec-49233b57793a None admin] PATH_INFO: `/v3/roles` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:01.521272 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-330d4767-d08b-42b4-baec-49233b57793a None admin] RBAC: Adding query filter params (name=member) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 21:59:01.521582 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-330d4767-d08b-42b4-baec-49233b57793a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:01.523025 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-330d4767-d08b-42b4-baec-49233b57793a None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:01.536574 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 168/347] 10.222.0.52 () {56 vars in 1101 bytes} [Wed Jul 13 21:59:01 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 21:59:01.537442 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330d4767-d08b-42b4-baec-49233b57793a None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:01.581770 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0563b4-15a4-4733-a2ad-e5a9564ba472 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:01.583645 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b0563b4-15a4-4733-a2ad-e5a9564ba472 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:01.619741 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:59:01.679858 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:59:01.694255 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 180/348] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Jul 13 21:59:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 358 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 13 21:59:01.695880 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0563b4-15a4-4733-a2ad-e5a9564ba472 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:01.733698 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-330d4767-d08b-42b4-baec-49233b57793a None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:01.810716 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b0563b4-15a4-4733-a2ad-e5a9564ba472 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:01.812319 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56914fc1-f02c-498a-b132-17c23722b849 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:01.813474 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56914fc1-f02c-498a-b132-17c23722b849 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:01.813775 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56914fc1-f02c-498a-b132-17c23722b849 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:01.815115 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56914fc1-f02c-498a-b132-17c23722b849 None admin] PATH_INFO: `/v3/projects/97aa9f72c7a54c9daa7e48288b4aab0f` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:01.821406 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56914fc1-f02c-498a-b132-17c23722b849 None admin] RBAC: Authorizing `identity:get_project(project_id=97aa9f72c7a54c9daa7e48288b4aab0f)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:01.823605 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56914fc1-f02c-498a-b132-17c23722b849 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:01.826927 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 169/349] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Jul 13 21:59:01 2022] GET /identity/v3/projects/97aa9f72c7a54c9daa7e48288b4aab0f => generated 298 bytes in 290 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 21:59:01.828163 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56914fc1-f02c-498a-b132-17c23722b849 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:01.858059 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9266f59c-2e65-4b64-b6e7-3f2828e31520 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:01.858976 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9266f59c-2e65-4b64-b6e7-3f2828e31520 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:01.859390 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9266f59c-2e65-4b64-b6e7-3f2828e31520 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:01.859798 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9266f59c-2e65-4b64-b6e7-3f2828e31520 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:01.933644 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:59:01.946610 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9266f59c-2e65-4b64-b6e7-3f2828e31520 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:01.948449 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9266f59c-2e65-4b64-b6e7-3f2828e31520 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:01.950479 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56914fc1-f02c-498a-b132-17c23722b849 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:01.970378 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:59:01.996155 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4819b0c-e559-4a74-9a76-65b70cfda959 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:01.997347 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4819b0c-e559-4a74-9a76-65b70cfda959 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:01.997784 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4819b0c-e559-4a74-9a76-65b70cfda959 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:01.998177 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4819b0c-e559-4a74-9a76-65b70cfda959 None admin] PATH_INFO: `/v3/projects/97aa9f72c7a54c9daa7e48288b4aab0f` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:01.999567 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4819b0c-e559-4a74-9a76-65b70cfda959 None admin] RBAC: Authorizing `identity:get_project(project_id=97aa9f72c7a54c9daa7e48288b4aab0f)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:02.001141 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4819b0c-e559-4a74-9a76-65b70cfda959 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:02.007676 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 170/350] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Jul 13 21:59:01 2022] GET /identity/v3/projects/97aa9f72c7a54c9daa7e48288b4aab0f => generated 298 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 21:59:02.008704 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879e4af5-0c36-46bd-a38a-30fd0a6e843a None None] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:02.009002 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879e4af5-0c36-46bd-a38a-30fd0a6e843a None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:02.009362 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879e4af5-0c36-46bd-a38a-30fd0a6e843a None None] PATH_INFO: `/` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:02.010748 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 171/351] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jul 13 21:59:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 21:59:02.011625 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879e4af5-0c36-46bd-a38a-30fd0a6e843a None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:02.073215 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 181/352] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Jul 13 21:59:01 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 378 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 13 21:59:02.075278 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532b995b-09e5-4e58-b2b9-c6aaa0e7fe0e None None] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:02.075760 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532b995b-09e5-4e58-b2b9-c6aaa0e7fe0e None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:02.076182 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532b995b-09e5-4e58-b2b9-c6aaa0e7fe0e None None] PATH_INFO: `/` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:02.077606 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 182/353] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jul 13 21:59:02 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 21:59:02.078572 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532b995b-09e5-4e58-b2b9-c6aaa0e7fe0e None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:02.134626 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-879e4af5-0c36-46bd-a38a-30fd0a6e843a None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:02.180068 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a35b7961-2706-4156-b6fc-3ee38a1833b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:02.181055 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35b7961-2706-4156-b6fc-3ee38a1833b6 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:02.181407 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35b7961-2706-4156-b6fc-3ee38a1833b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:02.181790 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a35b7961-2706-4156-b6fc-3ee38a1833b6 None admin] PATH_INFO: `/v3/groups/061e2f5095f2431198cdcc9162dee432` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:02.186564 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:59:02.189212 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a35b7961-2706-4156-b6fc-3ee38a1833b6 None admin] RBAC: Authorizing `identity:get_group(group_id=061e2f5095f2431198cdcc9162dee432)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:02.191594 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a35b7961-2706-4156-b6fc-3ee38a1833b6 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:02.193631 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 172/354] 10.222.0.52 () {56 vars in 1180 bytes} [Wed Jul 13 21:59:02 2022] GET /identity/v3/groups/061e2f5095f2431198cdcc9162dee432 => generated 226 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 21:59:02.195868 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabc7064-3421-4699-a465-a27f3ed34ef1 None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:02.196332 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabc7064-3421-4699-a465-a27f3ed34ef1 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:02.196824 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eabc7064-3421-4699-a465-a27f3ed34ef1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:02.198758 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532b995b-09e5-4e58-b2b9-c6aaa0e7fe0e None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:02.217375 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-eabc7064-3421-4699-a465-a27f3ed34ef1 None None] MFA Rules not processed for user `cc99c93da5074cf5a48bc2aa9a92c5e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:59:02.218655 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-eabc7064-3421-4699-a465-a27f3ed34ef1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:59:02.240024 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0f9c7f-eebf-49f0-a1fe-e6a31e01daaf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:02.240905 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0f9c7f-eebf-49f0-a1fe-e6a31e01daaf None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:02.241186 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0f9c7f-eebf-49f0-a1fe-e6a31e01daaf None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:02.241458 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d0f9c7f-eebf-49f0-a1fe-e6a31e01daaf None admin] PATH_INFO: `/v3/groups/061e2f5095f2431198cdcc9162dee432` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:02.242665 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0f9c7f-eebf-49f0-a1fe-e6a31e01daaf None admin] RBAC: Authorizing `identity:get_group(group_id=061e2f5095f2431198cdcc9162dee432)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:02.244146 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d0f9c7f-eebf-49f0-a1fe-e6a31e01daaf None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:02.246329 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 183/355] 10.222.0.52 () {56 vars in 1180 bytes} [Wed Jul 13 21:59:02 2022] GET /identity/v3/groups/061e2f5095f2431198cdcc9162dee432 => generated 226 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 21:59:02.247311 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0f9c7f-eebf-49f0-a1fe-e6a31e01daaf None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:02.275227 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:59:02.305920 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 173/356] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jul 13 21:59:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 13 21:59:02.307755 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9731b037-2b67-45ca-bb9b-433e328be159 None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:02.307755 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9731b037-2b67-45ca-bb9b-433e328be159 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:02.307957 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9731b037-2b67-45ca-bb9b-433e328be159 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:02.325543 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-9731b037-2b67-45ca-bb9b-433e328be159 None None] MFA Rules not processed for user `cc99c93da5074cf5a48bc2aa9a92c5e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:59:02.326866 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-9731b037-2b67-45ca-bb9b-433e328be159 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:59:02.373720 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d0f9c7f-eebf-49f0-a1fe-e6a31e01daaf None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:02.415018 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 174/357] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jul 13 21:59:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 13 21:59:02.416488 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9731b037-2b67-45ca-bb9b-433e328be159 None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:02.417252 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61294e1-7b63-412d-916a-00ae681651bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:02.418439 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61294e1-7b63-412d-916a-00ae681651bd None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:02.418439 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61294e1-7b63-412d-916a-00ae681651bd None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:02.418752 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61294e1-7b63-412d-916a-00ae681651bd None admin] PATH_INFO: `/v3/projects/4896b13c732d49d09947c2baaf9dc04d` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:02.420019 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d61294e1-7b63-412d-916a-00ae681651bd None admin] RBAC: Authorizing `identity:get_project(project_id=4896b13c732d49d09947c2baaf9dc04d)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:02.423090 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d61294e1-7b63-412d-916a-00ae681651bd None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:02.425197 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 184/358] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Jul 13 21:59:02 2022] GET /identity/v3/projects/4896b13c732d49d09947c2baaf9dc04d => generated 302 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 21:59:02.426033 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61294e1-7b63-412d-916a-00ae681651bd None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:02.455022 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:59:02.544778 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9731b037-2b67-45ca-bb9b-433e328be159 None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:02.551186 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61294e1-7b63-412d-916a-00ae681651bd None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:02.559390 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:59:02.593492 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964ed54f-e366-45c8-b99f-06ae32fb2c57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:02.594162 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964ed54f-e366-45c8-b99f-06ae32fb2c57 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:02.594445 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964ed54f-e366-45c8-b99f-06ae32fb2c57 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:02.594748 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-964ed54f-e366-45c8-b99f-06ae32fb2c57 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:02.595590 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-964ed54f-e366-45c8-b99f-06ae32fb2c57 None admin] RBAC: Adding query filter params (group.id=061e2f5095f2431198cdcc9162dee432, role.id=db84c6c897534af1a60b9ce62532f383, scope.project.id=97aa9f72c7a54c9daa7e48288b4aab0f) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 21:59:02.595904 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-964ed54f-e366-45c8-b99f-06ae32fb2c57 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:02.595995 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa84f738-7997-4685-864d-2248719026ee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:02.596642 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa84f738-7997-4685-864d-2248719026ee None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:02.596916 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa84f738-7997-4685-864d-2248719026ee None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:02.597213 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa84f738-7997-4685-864d-2248719026ee None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:02.597365 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-964ed54f-e366-45c8-b99f-06ae32fb2c57 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:02.598047 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa84f738-7997-4685-864d-2248719026ee None admin] RBAC: Adding query filter params (group.id=061e2f5095f2431198cdcc9162dee432, role.id=2a5e86976394498abb6310697cdb3f8b, scope.project.id=97aa9f72c7a54c9daa7e48288b4aab0f) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 21:59:02.598350 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa84f738-7997-4685-864d-2248719026ee None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:02.599801 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa84f738-7997-4685-864d-2248719026ee None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:02.603314 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 185/359] 10.222.0.52 () {56 vars in 1376 bytes} [Wed Jul 13 21:59:02 2022] GET /identity/v3/role_assignments?group.id=061e2f5095f2431198cdcc9162dee432&scope.project.id=97aa9f72c7a54c9daa7e48288b4aab0f&role.id=db84c6c897534af1a60b9ce62532f383 => generated 260 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 21:59:02.604350 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964ed54f-e366-45c8-b99f-06ae32fb2c57 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:02.605607 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 175/360] 10.222.0.52 () {56 vars in 1376 bytes} [Wed Jul 13 21:59:02 2022] GET /identity/v3/role_assignments?group.id=061e2f5095f2431198cdcc9162dee432&scope.project.id=97aa9f72c7a54c9daa7e48288b4aab0f&role.id=2a5e86976394498abb6310697cdb3f8b => generated 260 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 21:59:02.606590 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa84f738-7997-4685-864d-2248719026ee None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:02.720192 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:59:02.738636 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-964ed54f-e366-45c8-b99f-06ae32fb2c57 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:02.743718 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa84f738-7997-4685-864d-2248719026ee None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:02.785993 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ed25408-a952-4c5d-8198-313af694af17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:02.787264 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed25408-a952-4c5d-8198-313af694af17 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:02.787750 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed25408-a952-4c5d-8198-313af694af17 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:02.788239 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ed25408-a952-4c5d-8198-313af694af17 None admin] PATH_INFO: `/v3/roles/reader` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:02.790172 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1c23ed-c653-47a0-9ca6-4e59de4652ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:02.791401 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1c23ed-c653-47a0-9ca6-4e59de4652ca None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:02.791679 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1c23ed-c653-47a0-9ca6-4e59de4652ca None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:02.791946 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa1c23ed-c653-47a0-9ca6-4e59de4652ca None admin] PATH_INFO: `/v3/groups/061e2f5095f2431198cdcc9162dee432` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:02.793659 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa1c23ed-c653-47a0-9ca6-4e59de4652ca None admin] RBAC: Authorizing `identity:get_group(group_id=061e2f5095f2431198cdcc9162dee432)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:02.795718 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa1c23ed-c653-47a0-9ca6-4e59de4652ca None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:02.797973 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 176/361] 10.222.0.52 () {56 vars in 1180 bytes} [Wed Jul 13 21:59:02 2022] GET /identity/v3/groups/061e2f5095f2431198cdcc9162dee432 => generated 226 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 21:59:02.799569 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1c23ed-c653-47a0-9ca6-4e59de4652ca None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:02.803415 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ed25408-a952-4c5d-8198-313af694af17 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 21:59:02.812159 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application [None req-1ed25408-a952-4c5d-8198-313af694af17 None admin] Could not find role: reader.: keystone.exception.RoleNotFound: Could not find role: reader. Jul 13 21:59:02.812159 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 21:59:02.812159 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 21:59:02.812159 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 21:59:02.812159 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 21:59:02.812159 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 21:59:02.812159 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 21:59:02.812159 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 21:59:02.812159 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 21:59:02.812159 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 21:59:02.812159 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 21:59:02.812159 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 21:59:02.812159 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 21:59:02.812159 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 21:59:02.812159 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 21:59:02.812159 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 21:59:02.813112 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 21:59:02.813112 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 21:59:02.813112 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 21:59:02.813112 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 21:59:02.813112 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 21:59:02.813112 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 21:59:02.813112 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 21:59:02.813112 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 21:59:02.813112 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 21:59:02.813112 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 21:59:02.813112 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 21:59:02.813112 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application with Lock( Jul 13 21:59:02.813112 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 21:59:02.813112 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._enter() Jul 13 21:59:02.813112 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 21:59:02.813112 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 21:59:02.813901 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 21:59:02.813901 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.creator() Jul 13 21:59:02.813901 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 21:59:02.813901 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application created_value = creator( Jul 13 21:59:02.813901 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 21:59:02.813901 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 21:59:02.813901 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 21:59:02.813901 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 21:59:02.813901 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 21:59:02.813901 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 21:59:02.813901 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: reader. Jul 13 21:59:02.813901 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Jul 13 21:59:02.813901 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 186/362] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Jul 13 21:59:02 2022] GET /identity/v3/roles/reader => generated 84 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 21:59:02.818577 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d6f0a7-5926-427a-aada-f71af5160989 None None] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:02.818577 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d6f0a7-5926-427a-aada-f71af5160989 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:02.818577 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d6f0a7-5926-427a-aada-f71af5160989 None None] PATH_INFO: `/` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:02.818577 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 187/363] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jul 13 21:59:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 21:59:02.818577 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d6f0a7-5926-427a-aada-f71af5160989 None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:02.882860 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:59:02.923605 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa1c23ed-c653-47a0-9ca6-4e59de4652ca None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:02.941365 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d6f0a7-5926-427a-aada-f71af5160989 None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:02.964835 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4da1e9c3-46c5-45d5-bf8c-c535cc5b9ac2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:02.966213 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da1e9c3-46c5-45d5-bf8c-c535cc5b9ac2 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:02.966724 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da1e9c3-46c5-45d5-bf8c-c535cc5b9ac2 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:02.967192 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4da1e9c3-46c5-45d5-bf8c-c535cc5b9ac2 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:02.979604 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4da1e9c3-46c5-45d5-bf8c-c535cc5b9ac2 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 21:59:02.983212 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb064a0c-bbb7-4c05-a57b-98a72c8c07fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:02.983876 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb064a0c-bbb7-4c05-a57b-98a72c8c07fe None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:02.984153 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb064a0c-bbb7-4c05-a57b-98a72c8c07fe None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:02.984420 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb064a0c-bbb7-4c05-a57b-98a72c8c07fe None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:02.985263 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb064a0c-bbb7-4c05-a57b-98a72c8c07fe None admin] RBAC: Adding query filter params (group.id=061e2f5095f2431198cdcc9162dee432, role.id=2a5e86976394498abb6310697cdb3f8b, scope.project.id=4896b13c732d49d09947c2baaf9dc04d) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 21:59:02.985570 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb064a0c-bbb7-4c05-a57b-98a72c8c07fe None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:02.987055 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb064a0c-bbb7-4c05-a57b-98a72c8c07fe None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:02.990404 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application [None req-4da1e9c3-46c5-45d5-bf8c-c535cc5b9ac2 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 13 21:59:02.990404 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 21:59:02.990404 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 21:59:02.990404 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 21:59:02.990404 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 21:59:02.990404 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 21:59:02.990404 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 21:59:02.990404 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 21:59:02.990404 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 21:59:02.990404 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 21:59:02.990404 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 21:59:02.990404 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 21:59:02.990404 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 21:59:02.990404 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 21:59:02.990404 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 21:59:02.990404 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 21:59:02.991324 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 21:59:02.991324 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 21:59:02.991324 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 21:59:02.991324 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 21:59:02.991324 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 21:59:02.991324 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 21:59:02.991324 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 21:59:02.991324 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 21:59:02.991324 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 21:59:02.991324 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 21:59:02.991324 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 21:59:02.991324 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application with Lock( Jul 13 21:59:02.991324 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 21:59:02.991324 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._enter() Jul 13 21:59:02.991324 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 21:59:02.991324 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 21:59:02.992141 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 21:59:02.992141 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.creator() Jul 13 21:59:02.992141 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 21:59:02.992141 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application created_value = creator( Jul 13 21:59:02.992141 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 21:59:02.992141 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 21:59:02.992141 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 21:59:02.992141 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 21:59:02.992141 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 21:59:02.992141 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 21:59:02.992141 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 13 21:59:02.992141 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Jul 13 21:59:02.993357 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 188/364] 10.222.0.52 () {56 vars in 1376 bytes} [Wed Jul 13 21:59:02 2022] GET /identity/v3/role_assignments?group.id=061e2f5095f2431198cdcc9162dee432&scope.project.id=4896b13c732d49d09947c2baaf9dc04d&role.id=2a5e86976394498abb6310697cdb3f8b => generated 260 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 21:59:02.994309 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb064a0c-bbb7-4c05-a57b-98a72c8c07fe None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:02.995192 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 177/365] 10.222.0.52 () {56 vars in 1114 bytes} [Wed Jul 13 21:59:02 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 21:59:02.996922 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f28c613-fd4f-4ecf-bb10-d1259cda27d5 None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:02.997508 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f28c613-fd4f-4ecf-bb10-d1259cda27d5 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:02.998078 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f28c613-fd4f-4ecf-bb10-d1259cda27d5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:03.013354 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:59:03.021083 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-4f28c613-fd4f-4ecf-bb10-d1259cda27d5 None None] MFA Rules not processed for user `cc99c93da5074cf5a48bc2aa9a92c5e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:59:03.022711 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-4f28c613-fd4f-4ecf-bb10-d1259cda27d5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:59:03.108679 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 178/366] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jul 13 21:59:02 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 13 21:59:03.110017 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f28c613-fd4f-4ecf-bb10-d1259cda27d5 None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:03.126062 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb064a0c-bbb7-4c05-a57b-98a72c8c07fe None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:03.145256 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:59:03.164706 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24956825-006b-4b5a-9453-d8b16f8da7b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:03.165383 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24956825-006b-4b5a-9453-d8b16f8da7b7 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:03.165661 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24956825-006b-4b5a-9453-d8b16f8da7b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:03.165932 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24956825-006b-4b5a-9453-d8b16f8da7b7 None admin] PATH_INFO: `/v3/roles` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:03.167059 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24956825-006b-4b5a-9453-d8b16f8da7b7 None admin] RBAC: Adding query filter params (name=reader) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 21:59:03.167352 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24956825-006b-4b5a-9453-d8b16f8da7b7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:03.168736 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24956825-006b-4b5a-9453-d8b16f8da7b7 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:03.179040 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 189/367] 10.222.0.52 () {56 vars in 1101 bytes} [Wed Jul 13 21:59:02 2022] GET /identity/v3/roles?name=reader => generated 341 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 21:59:03.179923 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24956825-006b-4b5a-9453-d8b16f8da7b7 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:03.235191 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f28c613-fd4f-4ecf-bb10-d1259cda27d5 None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:03.271313 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:59:03.277095 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7117c72-b5da-41cd-9f62-ddf14d8e7e38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:03.278164 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7117c72-b5da-41cd-9f62-ddf14d8e7e38 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:03.278714 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7117c72-b5da-41cd-9f62-ddf14d8e7e38 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:03.279175 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7117c72-b5da-41cd-9f62-ddf14d8e7e38 None admin] PATH_INFO: `/v3/roles` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:03.280134 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7117c72-b5da-41cd-9f62-ddf14d8e7e38 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 21:59:03.280631 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7117c72-b5da-41cd-9f62-ddf14d8e7e38 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:03.282209 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7117c72-b5da-41cd-9f62-ddf14d8e7e38 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:03.292433 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 179/368] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Jul 13 21:59:03 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 21:59:03.293816 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7117c72-b5da-41cd-9f62-ddf14d8e7e38 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:03.296473 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24956825-006b-4b5a-9453-d8b16f8da7b7 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:03.334321 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782dc44c-d7f4-4790-93f4-041bfb9978f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:03.335187 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782dc44c-d7f4-4790-93f4-041bfb9978f7 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:03.335464 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782dc44c-d7f4-4790-93f4-041bfb9978f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:03.335736 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-782dc44c-d7f4-4790-93f4-041bfb9978f7 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:03.348447 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-782dc44c-d7f4-4790-93f4-041bfb9978f7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 21:59:03.357593 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application [None req-782dc44c-d7f4-4790-93f4-041bfb9978f7 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 13 21:59:03.357593 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 21:59:03.357593 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 21:59:03.357593 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 21:59:03.357593 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 21:59:03.357593 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 21:59:03.357593 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 21:59:03.357593 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 21:59:03.357593 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 21:59:03.357593 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 21:59:03.357593 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 21:59:03.357593 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 21:59:03.357593 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 21:59:03.357593 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 21:59:03.357593 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 21:59:03.357593 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 21:59:03.358619 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 21:59:03.358619 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 21:59:03.358619 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 21:59:03.358619 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 21:59:03.358619 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 21:59:03.358619 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 21:59:03.358619 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 21:59:03.358619 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 21:59:03.358619 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 21:59:03.358619 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 21:59:03.358619 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 21:59:03.358619 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application with Lock( Jul 13 21:59:03.358619 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 21:59:03.358619 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._enter() Jul 13 21:59:03.358619 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 21:59:03.358619 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 21:59:03.360083 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 21:59:03.360083 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.creator() Jul 13 21:59:03.360083 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 21:59:03.360083 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application created_value = creator( Jul 13 21:59:03.360083 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 21:59:03.360083 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 21:59:03.360083 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 21:59:03.360083 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 21:59:03.360083 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 21:59:03.360083 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 21:59:03.360083 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 13 21:59:03.360083 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Jul 13 21:59:03.360083 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 190/369] 10.222.0.52 () {56 vars in 1096 bytes} [Wed Jul 13 21:59:03 2022] GET /identity/v3/roles/admin => generated 83 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 21:59:03.360083 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782dc44c-d7f4-4790-93f4-041bfb9978f7 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:03.402276 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:59:03.413216 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7117c72-b5da-41cd-9f62-ddf14d8e7e38 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:03.448828 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e5a757-6528-416a-b2c8-7a29dd54a6d4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:03.450603 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e5a757-6528-416a-b2c8-7a29dd54a6d4 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:03.451077 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e5a757-6528-416a-b2c8-7a29dd54a6d4 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:03.451584 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e5a757-6528-416a-b2c8-7a29dd54a6d4 None admin] PATH_INFO: `/v3/users/54013c244a4e4e46b8ae8d0d714138bf` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:03.453253 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e5a757-6528-416a-b2c8-7a29dd54a6d4 None admin] RBAC: Authorizing `identity:get_user(user_id=54013c244a4e4e46b8ae8d0d714138bf)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:03.455009 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e5a757-6528-416a-b2c8-7a29dd54a6d4 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:03.456927 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 180/370] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Jul 13 21:59:03 2022] GET /identity/v3/users/54013c244a4e4e46b8ae8d0d714138bf => generated 293 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 21:59:03.458132 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e5a757-6528-416a-b2c8-7a29dd54a6d4 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:03.481219 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-782dc44c-d7f4-4790-93f4-041bfb9978f7 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:03.518043 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19098979-35ec-46a4-811f-742655e7e9fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:03.518631 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19098979-35ec-46a4-811f-742655e7e9fb None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:03.518915 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19098979-35ec-46a4-811f-742655e7e9fb None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:03.519187 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19098979-35ec-46a4-811f-742655e7e9fb None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:03.542058 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:59:03.582076 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e5a757-6528-416a-b2c8-7a29dd54a6d4 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:03.599430 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19098979-35ec-46a4-811f-742655e7e9fb None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:03.599430 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19098979-35ec-46a4-811f-742655e7e9fb None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:03.634537 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db944c6b-d491-455d-95d6-9002b92cef4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:03.635602 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db944c6b-d491-455d-95d6-9002b92cef4a None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:03.636076 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db944c6b-d491-455d-95d6-9002b92cef4a None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:03.636559 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db944c6b-d491-455d-95d6-9002b92cef4a None admin] PATH_INFO: `/v3/roles` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:03.638456 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db944c6b-d491-455d-95d6-9002b92cef4a None admin] RBAC: Adding query filter params (name=admin) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 21:59:03.638991 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db944c6b-d491-455d-95d6-9002b92cef4a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:03.640576 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db944c6b-d491-455d-95d6-9002b92cef4a None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:03.652200 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 181/371] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Jul 13 21:59:03 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 21:59:03.653839 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db944c6b-d491-455d-95d6-9002b92cef4a None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:03.685902 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:59:03.733858 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 191/372] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Jul 13 21:59:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 374 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 13 21:59:03.734743 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19098979-35ec-46a4-811f-742655e7e9fb None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:03.801760 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:59:03.853469 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db944c6b-d491-455d-95d6-9002b92cef4a None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:03.886752 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19098979-35ec-46a4-811f-742655e7e9fb None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:03.905997 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51247a17-e13b-4c68-9acf-7002fa4f81aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:03.906727 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51247a17-e13b-4c68-9acf-7002fa4f81aa None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:03.907013 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51247a17-e13b-4c68-9acf-7002fa4f81aa None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:03.907290 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51247a17-e13b-4c68-9acf-7002fa4f81aa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:03.908145 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51247a17-e13b-4c68-9acf-7002fa4f81aa None admin] RBAC: Adding query filter params (role.id=abb8511cb7d54b249e85bbc7507f0317, user.id=54013c244a4e4e46b8ae8d0d714138bf, scope.system=all) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 21:59:03.908459 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51247a17-e13b-4c68-9acf-7002fa4f81aa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:03.909926 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51247a17-e13b-4c68-9acf-7002fa4f81aa None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:03.922933 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 182/373] 10.222.0.52 () {56 vars in 1308 bytes} [Wed Jul 13 21:59:03 2022] GET /identity/v3/role_assignments?user.id=54013c244a4e4e46b8ae8d0d714138bf&scope.system=all&role.id=abb8511cb7d54b249e85bbc7507f0317 => generated 510 bytes in 270 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 21:59:03.923775 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51247a17-e13b-4c68-9acf-7002fa4f81aa None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:03.932335 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab13120-5591-48eb-88be-7423c462babe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:03.933135 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab13120-5591-48eb-88be-7423c462babe None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:03.933417 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab13120-5591-48eb-88be-7423c462babe None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:03.934206 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ab13120-5591-48eb-88be-7423c462babe None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:04.067525 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab13120-5591-48eb-88be-7423c462babe None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:04.068946 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ab13120-5591-48eb-88be-7423c462babe None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:04.095075 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:59:04.128341 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:59:04.147209 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51247a17-e13b-4c68-9acf-7002fa4f81aa None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:04.194874 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deef351c-bcaf-40ff-9c81-ee25d1674200 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:04.196133 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deef351c-bcaf-40ff-9c81-ee25d1674200 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:04.196714 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deef351c-bcaf-40ff-9c81-ee25d1674200 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:04.197175 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deef351c-bcaf-40ff-9c81-ee25d1674200 None admin] PATH_INFO: `/v3/projects/4896b13c732d49d09947c2baaf9dc04d` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:04.198746 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deef351c-bcaf-40ff-9c81-ee25d1674200 None admin] RBAC: Authorizing `identity:get_project(project_id=4896b13c732d49d09947c2baaf9dc04d)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:04.200800 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deef351c-bcaf-40ff-9c81-ee25d1674200 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:04.203126 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 183/374] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Jul 13 21:59:03 2022] GET /identity/v3/projects/4896b13c732d49d09947c2baaf9dc04d => generated 302 bytes in 279 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 21:59:04.209049 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deef351c-bcaf-40ff-9c81-ee25d1674200 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:04.237439 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 192/375] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Jul 13 21:59:03 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 503 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 13 21:59:04.242777 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab13120-5591-48eb-88be-7423c462babe None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:04.360568 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deef351c-bcaf-40ff-9c81-ee25d1674200 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:04.383419 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ab13120-5591-48eb-88be-7423c462babe None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:04.410834 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f980ec-cc29-4dc3-b625-b935523411b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:04.411708 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f980ec-cc29-4dc3-b625-b935523411b8 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:04.411983 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f980ec-cc29-4dc3-b625-b935523411b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:04.412248 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3f980ec-cc29-4dc3-b625-b935523411b8 None admin] PATH_INFO: `/v3/groups/061e2f5095f2431198cdcc9162dee432` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:04.415357 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f980ec-cc29-4dc3-b625-b935523411b8 None admin] RBAC: Authorizing `identity:get_group(group_id=061e2f5095f2431198cdcc9162dee432)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:04.416960 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3f980ec-cc29-4dc3-b625-b935523411b8 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:04.418888 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 184/376] 10.222.0.52 () {56 vars in 1180 bytes} [Wed Jul 13 21:59:04 2022] GET /identity/v3/groups/061e2f5095f2431198cdcc9162dee432 => generated 226 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 21:59:04.424855 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f980ec-cc29-4dc3-b625-b935523411b8 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:04.441455 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5f4905-efd7-4687-9b45-8e582ff2ec4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:04.442358 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5f4905-efd7-4687-9b45-8e582ff2ec4d None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:04.442732 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5f4905-efd7-4687-9b45-8e582ff2ec4d None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:04.443109 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5f4905-efd7-4687-9b45-8e582ff2ec4d None admin] PATH_INFO: `/v3/projects/0b902d068f594f05b4de544301553ded` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:04.444333 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5f4905-efd7-4687-9b45-8e582ff2ec4d None admin] RBAC: Authorizing `identity:get_project(project_id=0b902d068f594f05b4de544301553ded)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:04.445914 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5f4905-efd7-4687-9b45-8e582ff2ec4d None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:04.448938 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 193/377] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Jul 13 21:59:04 2022] GET /identity/v3/projects/0b902d068f594f05b4de544301553ded => generated 344 bytes in 206 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 21:59:04.454354 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:59:04.455705 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5f4905-efd7-4687-9b45-8e582ff2ec4d None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:04.483806 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:59:04.542333 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3f980ec-cc29-4dc3-b625-b935523411b8 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:04.583051 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c609781-fe86-4b12-b174-5d2da2c052e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:04.583708 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c609781-fe86-4b12-b174-5d2da2c052e0 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:04.583984 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c609781-fe86-4b12-b174-5d2da2c052e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:04.584253 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c609781-fe86-4b12-b174-5d2da2c052e0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:04.585093 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c609781-fe86-4b12-b174-5d2da2c052e0 None admin] RBAC: Adding query filter params (group.id=061e2f5095f2431198cdcc9162dee432, role.id=db84c6c897534af1a60b9ce62532f383, scope.project.id=4896b13c732d49d09947c2baaf9dc04d) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 21:59:04.585396 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c609781-fe86-4b12-b174-5d2da2c052e0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:04.587966 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c609781-fe86-4b12-b174-5d2da2c052e0 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:04.589461 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5f4905-efd7-4687-9b45-8e582ff2ec4d None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:04.597457 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 185/378] 10.222.0.52 () {56 vars in 1376 bytes} [Wed Jul 13 21:59:04 2022] GET /identity/v3/role_assignments?group.id=061e2f5095f2431198cdcc9162dee432&scope.project.id=4896b13c732d49d09947c2baaf9dc04d&role.id=db84c6c897534af1a60b9ce62532f383 => generated 260 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 21:59:04.652903 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42d6854-7532-4860-b328-8e53f4bdb74c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:04.653809 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42d6854-7532-4860-b328-8e53f4bdb74c None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:04.654090 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42d6854-7532-4860-b328-8e53f4bdb74c None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:04.654368 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42d6854-7532-4860-b328-8e53f4bdb74c None admin] PATH_INFO: `/v3/groups/1c51f1a766f44a05b929ec468d2d908c` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:04.668071 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42d6854-7532-4860-b328-8e53f4bdb74c None admin] RBAC: Authorizing `identity:get_group(group_id=1c51f1a766f44a05b929ec468d2d908c)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:04.669587 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b42d6854-7532-4860-b328-8e53f4bdb74c None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:04.671634 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 194/379] 10.222.0.52 () {56 vars in 1180 bytes} [Wed Jul 13 21:59:04 2022] GET /identity/v3/groups/1c51f1a766f44a05b929ec468d2d908c => generated 229 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 21:59:04.677476 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c609781-fe86-4b12-b174-5d2da2c052e0 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:04.796311 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:59:04.817659 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c609781-fe86-4b12-b174-5d2da2c052e0 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:04.853702 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed289a4f-a4b3-4e1a-bae3-e8ea73355e5d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:04.854369 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed289a4f-a4b3-4e1a-bae3-e8ea73355e5d None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:04.854675 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed289a4f-a4b3-4e1a-bae3-e8ea73355e5d None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:04.854941 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed289a4f-a4b3-4e1a-bae3-e8ea73355e5d None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:04.855786 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed289a4f-a4b3-4e1a-bae3-e8ea73355e5d None admin] RBAC: Adding query filter params (group.id=1c51f1a766f44a05b929ec468d2d908c, role.id=44efba2579f342df8a0789c59d163ec3, scope.project.id=0b902d068f594f05b4de544301553ded) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 21:59:04.856090 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed289a4f-a4b3-4e1a-bae3-e8ea73355e5d None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:04.857544 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed289a4f-a4b3-4e1a-bae3-e8ea73355e5d None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:04.867320 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 186/380] 10.222.0.52 () {56 vars in 1376 bytes} [Wed Jul 13 21:59:04 2022] GET /identity/v3/role_assignments?group.id=1c51f1a766f44a05b929ec468d2d908c&scope.project.id=0b902d068f594f05b4de544301553ded&role.id=44efba2579f342df8a0789c59d163ec3 => generated 260 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 21:59:05.390015 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393d1f7d-f09e-4c81-ac94-3c81106337de None None] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:05.390422 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393d1f7d-f09e-4c81-ac94-3c81106337de None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:05.390639 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-393d1f7d-f09e-4c81-ac94-3c81106337de None None] PATH_INFO: `/` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:05.392264 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 195/381] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jul 13 21:59:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 21:59:05.397803 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a7ff29-351f-4664-ac6a-92b4cebf1e19 None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:05.398085 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a7ff29-351f-4664-ac6a-92b4cebf1e19 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:05.398357 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3a7ff29-351f-4664-ac6a-92b4cebf1e19 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:05.421480 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-f3a7ff29-351f-4664-ac6a-92b4cebf1e19 None None] MFA Rules not processed for user `cc99c93da5074cf5a48bc2aa9a92c5e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:59:05.422701 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-f3a7ff29-351f-4664-ac6a-92b4cebf1e19 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:59:05.491308 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6008e7f8-092f-4a77-ac52-96085a60d94c None None] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:05.491854 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6008e7f8-092f-4a77-ac52-96085a60d94c None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:05.491854 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6008e7f8-092f-4a77-ac52-96085a60d94c None None] PATH_INFO: `/` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:05.493443 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 196/382] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jul 13 21:59:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 21:59:05.498881 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1517149-10ff-44d7-b2bb-f1004e21a8bc None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:05.499160 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1517149-10ff-44d7-b2bb-f1004e21a8bc None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:05.499433 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f1517149-10ff-44d7-b2bb-f1004e21a8bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:05.527818 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-f1517149-10ff-44d7-b2bb-f1004e21a8bc None None] MFA Rules not processed for user `cc99c93da5074cf5a48bc2aa9a92c5e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:59:05.529012 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-f1517149-10ff-44d7-b2bb-f1004e21a8bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:59:05.536287 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 187/383] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jul 13 21:59:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 139 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 13 21:59:05.563180 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a7ff29-351f-4664-ac6a-92b4cebf1e19 None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:05.646748 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:59:05.662237 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:59:05.673939 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3a7ff29-351f-4664-ac6a-92b4cebf1e19 None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:05.677952 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 197/384] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jul 13 21:59:05 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 13 21:59:05.710368 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d232355-39aa-42da-a881-ca054e765c71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:05.711457 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d232355-39aa-42da-a881-ca054e765c71 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:05.711596 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1517149-10ff-44d7-b2bb-f1004e21a8bc None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:05.712016 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d232355-39aa-42da-a881-ca054e765c71 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:05.712437 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d232355-39aa-42da-a881-ca054e765c71 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:05.723650 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d232355-39aa-42da-a881-ca054e765c71 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 21:59:05.732172 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application [None req-1d232355-39aa-42da-a881-ca054e765c71 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 13 21:59:05.732172 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 21:59:05.732172 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 21:59:05.732172 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 21:59:05.732172 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 21:59:05.732172 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 21:59:05.732172 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 21:59:05.732172 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 21:59:05.732172 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 21:59:05.732172 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 21:59:05.732172 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 21:59:05.732172 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 21:59:05.732172 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 21:59:05.732172 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 21:59:05.732172 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 21:59:05.732172 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 21:59:05.733329 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 21:59:05.733329 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 21:59:05.733329 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 21:59:05.733329 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 21:59:05.733329 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 21:59:05.733329 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 21:59:05.733329 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 21:59:05.733329 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 21:59:05.733329 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 21:59:05.733329 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 21:59:05.733329 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 21:59:05.733329 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application with Lock( Jul 13 21:59:05.733329 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 21:59:05.733329 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._enter() Jul 13 21:59:05.733329 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 21:59:05.733329 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 21:59:05.734380 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 21:59:05.734380 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.creator() Jul 13 21:59:05.734380 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 21:59:05.734380 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application created_value = creator( Jul 13 21:59:05.734380 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 21:59:05.734380 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 21:59:05.734380 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 21:59:05.734380 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 21:59:05.734380 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 21:59:05.734380 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 21:59:05.734380 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 13 21:59:05.734380 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Jul 13 21:59:05.736838 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 188/385] 10.222.0.52 () {56 vars in 1114 bytes} [Wed Jul 13 21:59:05 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 13 21:59:05.742729 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d232355-39aa-42da-a881-ca054e765c71 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:05.848391 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d232355-39aa-42da-a881-ca054e765c71 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:05.880295 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f1517149-10ff-44d7-b2bb-f1004e21a8bc None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:05.883473 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9341f4a-ee71-44cb-85dc-e7218adf8610 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:05.884347 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9341f4a-ee71-44cb-85dc-e7218adf8610 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:05.884819 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9341f4a-ee71-44cb-85dc-e7218adf8610 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:05.885251 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9341f4a-ee71-44cb-85dc-e7218adf8610 None admin] PATH_INFO: `/v3/roles` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:05.886173 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9341f4a-ee71-44cb-85dc-e7218adf8610 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 21:59:05.886664 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9341f4a-ee71-44cb-85dc-e7218adf8610 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:05.888198 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9341f4a-ee71-44cb-85dc-e7218adf8610 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:05.889955 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:59:05.898622 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 189/386] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Jul 13 21:59:05 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 21:59:05.904565 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9341f4a-ee71-44cb-85dc-e7218adf8610 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:05.940601 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5ceb1e-8ee2-43c2-816f-c3446e6fbaef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:05.941513 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5ceb1e-8ee2-43c2-816f-c3446e6fbaef None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:05.941973 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5ceb1e-8ee2-43c2-816f-c3446e6fbaef None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:05.942399 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d5ceb1e-8ee2-43c2-816f-c3446e6fbaef None admin] PATH_INFO: `/v3/roles/member` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:05.962500 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d5ceb1e-8ee2-43c2-816f-c3446e6fbaef None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 21:59:05.978848 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application [None req-7d5ceb1e-8ee2-43c2-816f-c3446e6fbaef None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 13 21:59:05.978848 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 21:59:05.978848 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 21:59:05.978848 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 21:59:05.978848 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 21:59:05.978848 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 21:59:05.978848 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 21:59:05.978848 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 21:59:05.978848 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 21:59:05.978848 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 21:59:05.978848 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 21:59:05.978848 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 21:59:05.978848 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 21:59:05.978848 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 21:59:05.978848 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 21:59:05.978848 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 21:59:05.979831 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 21:59:05.979831 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 21:59:05.979831 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 21:59:05.979831 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 21:59:05.979831 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 21:59:05.979831 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 21:59:05.979831 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 21:59:05.979831 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 21:59:05.979831 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 21:59:05.979831 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 21:59:05.979831 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 21:59:05.979831 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application with Lock( Jul 13 21:59:05.979831 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 21:59:05.979831 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._enter() Jul 13 21:59:05.979831 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 21:59:05.979831 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 21:59:05.980635 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 21:59:05.980635 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.creator() Jul 13 21:59:05.980635 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 21:59:05.980635 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application created_value = creator( Jul 13 21:59:05.980635 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 21:59:05.980635 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 21:59:05.980635 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 21:59:05.980635 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 21:59:05.980635 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 21:59:05.980635 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 21:59:05.980635 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 13 21:59:05.980635 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Jul 13 21:59:05.980635 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 198/387] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Jul 13 21:59:05 2022] GET /identity/v3/roles/member => generated 84 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 13 21:59:05.987760 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5ceb1e-8ee2-43c2-816f-c3446e6fbaef None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:06.007709 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9341f4a-ee71-44cb-85dc-e7218adf8610 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:06.045266 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6477ae-71bb-422d-a17d-7e4a2b727249 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:06.046450 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6477ae-71bb-422d-a17d-7e4a2b727249 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:06.046915 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6477ae-71bb-422d-a17d-7e4a2b727249 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:06.048437 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f6477ae-71bb-422d-a17d-7e4a2b727249 None admin] PATH_INFO: `/v3/groups/061e2f5095f2431198cdcc9162dee432` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:06.049842 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f6477ae-71bb-422d-a17d-7e4a2b727249 None admin] RBAC: Authorizing `identity:get_group(group_id=061e2f5095f2431198cdcc9162dee432)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:06.051500 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f6477ae-71bb-422d-a17d-7e4a2b727249 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:06.053621 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 190/388] 10.222.0.52 () {56 vars in 1180 bytes} [Wed Jul 13 21:59:05 2022] GET /identity/v3/groups/061e2f5095f2431198cdcc9162dee432 => generated 226 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 21:59:06.058139 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:59:06.059439 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6477ae-71bb-422d-a17d-7e4a2b727249 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:06.100237 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d5ceb1e-8ee2-43c2-816f-c3446e6fbaef None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:06.134732 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b81c7ba-ab43-4301-97e6-d0dc5a3d19c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:06.135621 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b81c7ba-ab43-4301-97e6-d0dc5a3d19c3 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:06.135911 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b81c7ba-ab43-4301-97e6-d0dc5a3d19c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:06.136171 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b81c7ba-ab43-4301-97e6-d0dc5a3d19c3 None admin] PATH_INFO: `/v3/roles` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:06.136927 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b81c7ba-ab43-4301-97e6-d0dc5a3d19c3 None admin] RBAC: Adding query filter params (name=member) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 21:59:06.137227 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b81c7ba-ab43-4301-97e6-d0dc5a3d19c3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:06.138637 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b81c7ba-ab43-4301-97e6-d0dc5a3d19c3 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:06.147325 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 199/389] 10.222.0.52 () {56 vars in 1101 bytes} [Wed Jul 13 21:59:05 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 21:59:06.148814 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303ed17e-bd72-41d2-860c-ee70966e2970 None None] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:06.149247 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303ed17e-bd72-41d2-860c-ee70966e2970 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:06.149663 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-303ed17e-bd72-41d2-860c-ee70966e2970 None None] PATH_INFO: `/` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:06.151421 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 200/390] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jul 13 21:59:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 21:59:06.154728 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:59:06.163536 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340e71ef-01ad-4587-9cd9-3318817951f4 None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:06.164112 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340e71ef-01ad-4587-9cd9-3318817951f4 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:06.164542 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-340e71ef-01ad-4587-9cd9-3318817951f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:06.186719 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-340e71ef-01ad-4587-9cd9-3318817951f4 None None] MFA Rules not processed for user `cc99c93da5074cf5a48bc2aa9a92c5e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:59:06.188397 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-340e71ef-01ad-4587-9cd9-3318817951f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:59:06.201041 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f6477ae-71bb-422d-a17d-7e4a2b727249 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:06.238603 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb5f647-fe43-4dbb-8787-219321b0150e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:06.239739 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb5f647-fe43-4dbb-8787-219321b0150e None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:06.240170 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb5f647-fe43-4dbb-8787-219321b0150e None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:06.240687 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6fb5f647-fe43-4dbb-8787-219321b0150e None admin] PATH_INFO: `/v3/projects/97aa9f72c7a54c9daa7e48288b4aab0f` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:06.242034 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb5f647-fe43-4dbb-8787-219321b0150e None admin] RBAC: Authorizing `identity:get_project(project_id=97aa9f72c7a54c9daa7e48288b4aab0f)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:06.243743 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6fb5f647-fe43-4dbb-8787-219321b0150e None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:06.246158 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 191/391] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Jul 13 21:59:06 2022] GET /identity/v3/projects/97aa9f72c7a54c9daa7e48288b4aab0f => generated 298 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 21:59:06.247387 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb5f647-fe43-4dbb-8787-219321b0150e None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:06.271974 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 201/392] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jul 13 21:59:06 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 13 21:59:06.278099 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340e71ef-01ad-4587-9cd9-3318817951f4 None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:06.330152 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:59:06.390048 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6fb5f647-fe43-4dbb-8787-219321b0150e None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:06.395195 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-340e71ef-01ad-4587-9cd9-3318817951f4 None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:06.427557 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:59:06.436127 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05f5865-c183-475c-b9c5-56c2c3f17628 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:06.437373 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05f5865-c183-475c-b9c5-56c2c3f17628 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:06.437901 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05f5865-c183-475c-b9c5-56c2c3f17628 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:06.438372 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f05f5865-c183-475c-b9c5-56c2c3f17628 None admin] PATH_INFO: `/v3/groups/061e2f5095f2431198cdcc9162dee432` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:06.440182 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05f5865-c183-475c-b9c5-56c2c3f17628 None admin] RBAC: Authorizing `identity:get_group(group_id=061e2f5095f2431198cdcc9162dee432)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:06.442684 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f05f5865-c183-475c-b9c5-56c2c3f17628 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:06.444571 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 192/393] 10.222.0.52 () {56 vars in 1180 bytes} [Wed Jul 13 21:59:06 2022] GET /identity/v3/groups/061e2f5095f2431198cdcc9162dee432 => generated 226 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 21:59:06.449552 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05f5865-c183-475c-b9c5-56c2c3f17628 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:06.450900 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87734e91-7b77-47a0-b9ff-c585bbf4eee5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:06.452154 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87734e91-7b77-47a0-b9ff-c585bbf4eee5 None admin] REQUEST_METHOD: `PUT` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:06.452431 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87734e91-7b77-47a0-b9ff-c585bbf4eee5 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:06.453734 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87734e91-7b77-47a0-b9ff-c585bbf4eee5 None admin] PATH_INFO: `/v3/projects/97aa9f72c7a54c9daa7e48288b4aab0f/groups/061e2f5095f2431198cdcc9162dee432/roles/db84c6c897534af1a60b9ce62532f383` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:06.460418 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87734e91-7b77-47a0-b9ff-c585bbf4eee5 None admin] RBAC: Authorizing `identity:create_grant(project_id=97aa9f72c7a54c9daa7e48288b4aab0f, group_id=061e2f5095f2431198cdcc9162dee432, role_id=db84c6c897534af1a60b9ce62532f383)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:06.462275 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87734e91-7b77-47a0-b9ff-c585bbf4eee5 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:06.482842 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 202/394] 10.222.0.52 () {58 vars in 1442 bytes} [Wed Jul 13 21:59:06 2022] PUT /identity/v3/projects/97aa9f72c7a54c9daa7e48288b4aab0f/groups/061e2f5095f2431198cdcc9162dee432/roles/db84c6c897534af1a60b9ce62532f383 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 21:59:06.484302 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87734e91-7b77-47a0-b9ff-c585bbf4eee5 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:06.605456 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87734e91-7b77-47a0-b9ff-c585bbf4eee5 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:06.617208 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:59:06.648474 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6096bb0-4b72-4582-b225-e810e90d0a21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:06.649795 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6096bb0-4b72-4582-b225-e810e90d0a21 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:06.650780 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6096bb0-4b72-4582-b225-e810e90d0a21 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:06.651389 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6096bb0-4b72-4582-b225-e810e90d0a21 None admin] PATH_INFO: `/v3/projects/97aa9f72c7a54c9daa7e48288b4aab0f` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:06.653070 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6096bb0-4b72-4582-b225-e810e90d0a21 None admin] RBAC: Authorizing `identity:get_project(project_id=97aa9f72c7a54c9daa7e48288b4aab0f)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:06.654802 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6096bb0-4b72-4582-b225-e810e90d0a21 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:06.657105 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 203/395] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Jul 13 21:59:06 2022] GET /identity/v3/projects/97aa9f72c7a54c9daa7e48288b4aab0f => generated 298 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 21:59:06.663298 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6096bb0-4b72-4582-b225-e810e90d0a21 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:06.694771 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f05f5865-c183-475c-b9c5-56c2c3f17628 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:06.754521 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78046278-910a-433e-ae89-3b16a60d24b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:06.755362 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78046278-910a-433e-ae89-3b16a60d24b1 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:06.755943 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78046278-910a-433e-ae89-3b16a60d24b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:06.756221 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-78046278-910a-433e-ae89-3b16a60d24b1 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:06.771594 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-78046278-910a-433e-ae89-3b16a60d24b1 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 21:59:06.783354 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6096bb0-4b72-4582-b225-e810e90d0a21 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:06.785725 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application [None req-78046278-910a-433e-ae89-3b16a60d24b1 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 13 21:59:06.785725 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 21:59:06.785725 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 21:59:06.785725 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 21:59:06.785725 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 21:59:06.785725 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 21:59:06.785725 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 21:59:06.785725 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 21:59:06.785725 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 21:59:06.785725 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 21:59:06.785725 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 21:59:06.785725 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 21:59:06.785725 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 21:59:06.785725 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 21:59:06.785725 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 21:59:06.785725 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 21:59:06.786673 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 21:59:06.786673 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 21:59:06.786673 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 21:59:06.786673 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 21:59:06.786673 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 21:59:06.786673 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 21:59:06.786673 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 21:59:06.786673 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 21:59:06.786673 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 21:59:06.786673 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 21:59:06.786673 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 21:59:06.786673 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application with Lock( Jul 13 21:59:06.786673 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 21:59:06.786673 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._enter() Jul 13 21:59:06.786673 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 21:59:06.786673 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 21:59:06.787587 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 21:59:06.787587 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.creator() Jul 13 21:59:06.787587 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 21:59:06.787587 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application created_value = creator( Jul 13 21:59:06.787587 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 21:59:06.787587 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 21:59:06.787587 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 21:59:06.787587 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 21:59:06.787587 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 21:59:06.787587 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 21:59:06.787587 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 13 21:59:06.787587 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Jul 13 21:59:06.787587 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 193/396] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Jul 13 21:59:06 2022] GET /identity/v3/roles/member => generated 84 bytes in 342 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 21:59:06.795927 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78046278-910a-433e-ae89-3b16a60d24b1 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:06.838847 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f12378a-16ac-4512-bd98-d5d74bb82f8b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:06.840482 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f12378a-16ac-4512-bd98-d5d74bb82f8b None admin] REQUEST_METHOD: `PUT` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:06.840482 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f12378a-16ac-4512-bd98-d5d74bb82f8b None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:06.840482 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f12378a-16ac-4512-bd98-d5d74bb82f8b None admin] PATH_INFO: `/v3/projects/97aa9f72c7a54c9daa7e48288b4aab0f/groups/061e2f5095f2431198cdcc9162dee432/roles/2a5e86976394498abb6310697cdb3f8b` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:06.844345 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f12378a-16ac-4512-bd98-d5d74bb82f8b None admin] RBAC: Authorizing `identity:create_grant(project_id=97aa9f72c7a54c9daa7e48288b4aab0f, group_id=061e2f5095f2431198cdcc9162dee432, role_id=2a5e86976394498abb6310697cdb3f8b)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:06.845826 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f12378a-16ac-4512-bd98-d5d74bb82f8b None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:06.863141 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 204/397] 10.222.0.52 () {58 vars in 1442 bytes} [Wed Jul 13 21:59:06 2022] PUT /identity/v3/projects/97aa9f72c7a54c9daa7e48288b4aab0f/groups/061e2f5095f2431198cdcc9162dee432/roles/2a5e86976394498abb6310697cdb3f8b => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 21:59:06.882901 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:59:06.946202 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-78046278-910a-433e-ae89-3b16a60d24b1 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:06.997912 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8c9c3c-ffaa-4b2b-82db-e10f6457a946 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:06.999057 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8c9c3c-ffaa-4b2b-82db-e10f6457a946 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:06.999565 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8c9c3c-ffaa-4b2b-82db-e10f6457a946 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:07.000053 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e8c9c3c-ffaa-4b2b-82db-e10f6457a946 None admin] PATH_INFO: `/v3/roles` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:07.001028 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8c9c3c-ffaa-4b2b-82db-e10f6457a946 None admin] RBAC: Adding query filter params (name=member) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 21:59:07.001547 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8c9c3c-ffaa-4b2b-82db-e10f6457a946 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:07.007402 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e8c9c3c-ffaa-4b2b-82db-e10f6457a946 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:07.017916 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 194/398] 10.222.0.52 () {56 vars in 1101 bytes} [Wed Jul 13 21:59:06 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 21:59:07.023173 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f12378a-16ac-4512-bd98-d5d74bb82f8b None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:07.070139 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:59:07.159229 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f12378a-16ac-4512-bd98-d5d74bb82f8b None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:07.196521 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f4cf87-25a7-4051-b58e-c2d7f57f8b06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:07.197357 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f4cf87-25a7-4051-b58e-c2d7f57f8b06 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:07.197632 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f4cf87-25a7-4051-b58e-c2d7f57f8b06 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:07.197906 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f4cf87-25a7-4051-b58e-c2d7f57f8b06 None admin] PATH_INFO: `/v3/groups/061e2f5095f2431198cdcc9162dee432` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:07.199615 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f4cf87-25a7-4051-b58e-c2d7f57f8b06 None admin] RBAC: Authorizing `identity:get_group(group_id=061e2f5095f2431198cdcc9162dee432)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:07.201069 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f4cf87-25a7-4051-b58e-c2d7f57f8b06 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:07.203091 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 205/399] 10.222.0.52 () {56 vars in 1180 bytes} [Wed Jul 13 21:59:07 2022] GET /identity/v3/groups/061e2f5095f2431198cdcc9162dee432 => generated 226 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 21:59:07.208285 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8c9c3c-ffaa-4b2b-82db-e10f6457a946 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:07.340554 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e8c9c3c-ffaa-4b2b-82db-e10f6457a946 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:07.382699 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:59:07.392209 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1792e27-d51b-43ed-a3ed-3f3350742fc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:07.393726 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1792e27-d51b-43ed-a3ed-3f3350742fc5 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:07.394517 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1792e27-d51b-43ed-a3ed-3f3350742fc5 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:07.394808 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1792e27-d51b-43ed-a3ed-3f3350742fc5 None admin] PATH_INFO: `/v3/projects/4896b13c732d49d09947c2baaf9dc04d` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:07.396424 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1792e27-d51b-43ed-a3ed-3f3350742fc5 None admin] RBAC: Authorizing `identity:get_project(project_id=4896b13c732d49d09947c2baaf9dc04d)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:07.397869 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1792e27-d51b-43ed-a3ed-3f3350742fc5 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:07.399974 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 195/400] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Jul 13 21:59:07 2022] GET /identity/v3/projects/4896b13c732d49d09947c2baaf9dc04d => generated 302 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 21:59:07.405917 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f4cf87-25a7-4051-b58e-c2d7f57f8b06 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:07.484000 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963b7b53-f858-4a74-a962-78d4a71b1611 None None] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:07.484524 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963b7b53-f858-4a74-a962-78d4a71b1611 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:07.484524 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-963b7b53-f858-4a74-a962-78d4a71b1611 None None] PATH_INFO: `/` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:07.485963 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 196/401] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jul 13 21:59:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 21:59:07.491111 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572c3f81-0b42-45db-9025-bc652fd2599a None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:07.491394 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572c3f81-0b42-45db-9025-bc652fd2599a None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:07.491672 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-572c3f81-0b42-45db-9025-bc652fd2599a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:07.509977 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-572c3f81-0b42-45db-9025-bc652fd2599a None None] MFA Rules not processed for user `cc99c93da5074cf5a48bc2aa9a92c5e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:59:07.511245 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-572c3f81-0b42-45db-9025-bc652fd2599a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:59:07.582996 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f4cf87-25a7-4051-b58e-c2d7f57f8b06 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:07.585397 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:59:07.620207 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42462f73-3710-42ed-8c93-fdf9f9832736 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:07.621107 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42462f73-3710-42ed-8c93-fdf9f9832736 None admin] REQUEST_METHOD: `PUT` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:07.621389 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42462f73-3710-42ed-8c93-fdf9f9832736 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:07.621663 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42462f73-3710-42ed-8c93-fdf9f9832736 None admin] PATH_INFO: `/v3/projects/4896b13c732d49d09947c2baaf9dc04d/groups/061e2f5095f2431198cdcc9162dee432/roles/2a5e86976394498abb6310697cdb3f8b` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:07.623726 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42462f73-3710-42ed-8c93-fdf9f9832736 None admin] RBAC: Authorizing `identity:create_grant(project_id=4896b13c732d49d09947c2baaf9dc04d, group_id=061e2f5095f2431198cdcc9162dee432, role_id=2a5e86976394498abb6310697cdb3f8b)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:07.625237 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42462f73-3710-42ed-8c93-fdf9f9832736 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:07.628787 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 197/402] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jul 13 21:59:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 138 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 13 21:59:07.630326 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16261071-b7c2-4a6c-86f0-50e2d3db3318 None None] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:07.630803 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16261071-b7c2-4a6c-86f0-50e2d3db3318 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:07.631240 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16261071-b7c2-4a6c-86f0-50e2d3db3318 None None] PATH_INFO: `/` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:07.632669 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 198/403] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jul 13 21:59:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 21:59:07.638407 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f12195c-afc2-4c4e-9675-99693b0c0939 None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:07.639002 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f12195c-afc2-4c4e-9675-99693b0c0939 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:07.639445 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f12195c-afc2-4c4e-9675-99693b0c0939 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:07.658522 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-9f12195c-afc2-4c4e-9675-99693b0c0939 None None] MFA Rules not processed for user `cc99c93da5074cf5a48bc2aa9a92c5e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:59:07.659844 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-9f12195c-afc2-4c4e-9675-99693b0c0939 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:59:07.662471 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 206/404] 10.222.0.52 () {58 vars in 1442 bytes} [Wed Jul 13 21:59:07 2022] PUT /identity/v3/projects/4896b13c732d49d09947c2baaf9dc04d/groups/061e2f5095f2431198cdcc9162dee432/roles/2a5e86976394498abb6310697cdb3f8b => generated 0 bytes in 257 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 21:59:07.664051 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42462f73-3710-42ed-8c93-fdf9f9832736 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:07.799927 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 199/405] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jul 13 21:59:07 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 13 21:59:07.842550 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f12195c-afc2-4c4e-9675-99693b0c0939 None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:07.845492 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42462f73-3710-42ed-8c93-fdf9f9832736 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:07.891295 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1105de45-a26d-495e-b8a4-d28ed3eac17b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:07.892955 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1105de45-a26d-495e-b8a4-d28ed3eac17b None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:07.893448 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1105de45-a26d-495e-b8a4-d28ed3eac17b None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:07.893918 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1105de45-a26d-495e-b8a4-d28ed3eac17b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:07.907343 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:59:07.911276 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1105de45-a26d-495e-b8a4-d28ed3eac17b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 21:59:07.923470 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application [None req-1105de45-a26d-495e-b8a4-d28ed3eac17b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 13 21:59:07.923470 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 21:59:07.923470 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 21:59:07.923470 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 21:59:07.923470 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 21:59:07.923470 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 21:59:07.923470 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 21:59:07.923470 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 21:59:07.923470 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 21:59:07.923470 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 21:59:07.923470 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 21:59:07.923470 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 21:59:07.923470 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 21:59:07.923470 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 21:59:07.923470 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 21:59:07.923470 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 21:59:07.924404 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 21:59:07.924404 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 21:59:07.924404 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 21:59:07.924404 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 21:59:07.924404 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 21:59:07.924404 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 21:59:07.924404 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 21:59:07.924404 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 21:59:07.924404 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 21:59:07.924404 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 21:59:07.924404 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 21:59:07.924404 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application with Lock( Jul 13 21:59:07.924404 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 21:59:07.924404 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._enter() Jul 13 21:59:07.924404 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 21:59:07.924404 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 21:59:07.925241 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 21:59:07.925241 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.creator() Jul 13 21:59:07.925241 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 21:59:07.925241 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application created_value = creator( Jul 13 21:59:07.925241 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 21:59:07.925241 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 21:59:07.925241 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 21:59:07.925241 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 21:59:07.925241 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 21:59:07.925241 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 21:59:07.925241 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 13 21:59:07.925241 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Jul 13 21:59:07.928112 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 207/406] 10.222.0.52 () {56 vars in 1096 bytes} [Wed Jul 13 21:59:07 2022] GET /identity/v3/roles/admin => generated 83 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 21:59:07.934999 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1105de45-a26d-495e-b8a4-d28ed3eac17b None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:07.963992 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f12195c-afc2-4c4e-9675-99693b0c0939 None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:07.976806 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:59:08.001182 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca107c8-7bd7-483e-83cd-25bd095b4320 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:08.002237 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca107c8-7bd7-483e-83cd-25bd095b4320 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:08.002649 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca107c8-7bd7-483e-83cd-25bd095b4320 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:08.003013 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ca107c8-7bd7-483e-83cd-25bd095b4320 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:08.019274 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ca107c8-7bd7-483e-83cd-25bd095b4320 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 21:59:08.028016 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application [None req-1ca107c8-7bd7-483e-83cd-25bd095b4320 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 13 21:59:08.028016 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 21:59:08.028016 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 21:59:08.028016 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 21:59:08.028016 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 21:59:08.028016 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 21:59:08.028016 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 21:59:08.028016 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 21:59:08.028016 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 21:59:08.028016 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 21:59:08.028016 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 21:59:08.028016 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 21:59:08.028016 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 21:59:08.028016 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 21:59:08.028016 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 21:59:08.028016 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 21:59:08.028935 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 21:59:08.028935 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 21:59:08.028935 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 21:59:08.028935 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 21:59:08.028935 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 21:59:08.028935 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 21:59:08.028935 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 21:59:08.028935 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 21:59:08.028935 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 21:59:08.028935 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 21:59:08.028935 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 21:59:08.028935 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application with Lock( Jul 13 21:59:08.028935 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 21:59:08.028935 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._enter() Jul 13 21:59:08.028935 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 21:59:08.028935 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 21:59:08.030151 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 21:59:08.030151 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.creator() Jul 13 21:59:08.030151 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 21:59:08.030151 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application created_value = creator( Jul 13 21:59:08.030151 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 21:59:08.030151 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 21:59:08.030151 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 21:59:08.030151 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 21:59:08.030151 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 21:59:08.030151 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 21:59:08.030151 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 13 21:59:08.030151 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Jul 13 21:59:08.030151 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 200/407] 10.222.0.52 () {56 vars in 1114 bytes} [Wed Jul 13 21:59:07 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 13 21:59:08.036214 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca107c8-7bd7-483e-83cd-25bd095b4320 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:08.102697 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1105de45-a26d-495e-b8a4-d28ed3eac17b None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:08.168603 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79620680-9d71-4f71-9255-bf6dbe85e473 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:08.169300 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79620680-9d71-4f71-9255-bf6dbe85e473 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:08.169577 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79620680-9d71-4f71-9255-bf6dbe85e473 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:08.169845 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79620680-9d71-4f71-9255-bf6dbe85e473 None admin] PATH_INFO: `/v3/roles` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:08.170629 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79620680-9d71-4f71-9255-bf6dbe85e473 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 21:59:08.170933 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79620680-9d71-4f71-9255-bf6dbe85e473 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:08.172559 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79620680-9d71-4f71-9255-bf6dbe85e473 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:08.192776 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ca107c8-7bd7-483e-83cd-25bd095b4320 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:08.200920 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 208/408] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Jul 13 21:59:07 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 262 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 21:59:08.209088 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79620680-9d71-4f71-9255-bf6dbe85e473 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:08.218811 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:59:08.257007 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdce9aa0-7ae3-44ee-9014-dda5b92cc791 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:08.257681 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdce9aa0-7ae3-44ee-9014-dda5b92cc791 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:08.257956 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdce9aa0-7ae3-44ee-9014-dda5b92cc791 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:08.258224 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdce9aa0-7ae3-44ee-9014-dda5b92cc791 None admin] PATH_INFO: `/v3/roles` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:08.259013 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdce9aa0-7ae3-44ee-9014-dda5b92cc791 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 21:59:08.259313 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdce9aa0-7ae3-44ee-9014-dda5b92cc791 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:08.260836 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdce9aa0-7ae3-44ee-9014-dda5b92cc791 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:08.280653 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 201/409] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Jul 13 21:59:08 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 246 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 21:59:08.286396 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdce9aa0-7ae3-44ee-9014-dda5b92cc791 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:08.321216 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:59:08.341243 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79620680-9d71-4f71-9255-bf6dbe85e473 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:08.381327 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c29112-b2be-49f5-a16f-3262200d881a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:08.382167 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c29112-b2be-49f5-a16f-3262200d881a None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:08.382442 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c29112-b2be-49f5-a16f-3262200d881a None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:08.382789 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c29112-b2be-49f5-a16f-3262200d881a None admin] PATH_INFO: `/v3/groups/1c51f1a766f44a05b929ec468d2d908c` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:08.383967 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c29112-b2be-49f5-a16f-3262200d881a None admin] RBAC: Authorizing `identity:get_group(group_id=1c51f1a766f44a05b929ec468d2d908c)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:08.386297 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c29112-b2be-49f5-a16f-3262200d881a None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:08.392587 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 209/410] 10.222.0.52 () {56 vars in 1180 bytes} [Wed Jul 13 21:59:08 2022] GET /identity/v3/groups/1c51f1a766f44a05b929ec468d2d908c => generated 229 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 21:59:08.393785 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c29112-b2be-49f5-a16f-3262200d881a None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:08.516050 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c29112-b2be-49f5-a16f-3262200d881a None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:08.535609 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdce9aa0-7ae3-44ee-9014-dda5b92cc791 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:08.564381 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d593679-4d30-4d4d-a903-8a59debc063a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:08.565274 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d593679-4d30-4d4d-a903-8a59debc063a None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:08.565556 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d593679-4d30-4d4d-a903-8a59debc063a None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:08.565920 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d593679-4d30-4d4d-a903-8a59debc063a None admin] PATH_INFO: `/v3/projects/0b902d068f594f05b4de544301553ded` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:08.567219 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d593679-4d30-4d4d-a903-8a59debc063a None admin] RBAC: Authorizing `identity:get_project(project_id=0b902d068f594f05b4de544301553ded)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:08.568794 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d593679-4d30-4d4d-a903-8a59debc063a None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:08.570559 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 210/411] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Jul 13 21:59:08 2022] GET /identity/v3/projects/0b902d068f594f05b4de544301553ded => generated 344 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 21:59:08.577492 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d593679-4d30-4d4d-a903-8a59debc063a None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:08.603498 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:59:08.619298 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a895f0ac-4156-4014-b9bb-1a068743a0a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:08.620330 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a895f0ac-4156-4014-b9bb-1a068743a0a8 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:08.620621 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a895f0ac-4156-4014-b9bb-1a068743a0a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:08.620899 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a895f0ac-4156-4014-b9bb-1a068743a0a8 None admin] PATH_INFO: `/v3/groups/061e2f5095f2431198cdcc9162dee432` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:08.622172 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a895f0ac-4156-4014-b9bb-1a068743a0a8 None admin] RBAC: Authorizing `identity:get_group(group_id=061e2f5095f2431198cdcc9162dee432)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:08.623666 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a895f0ac-4156-4014-b9bb-1a068743a0a8 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:08.625656 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 202/412] 10.222.0.52 () {56 vars in 1180 bytes} [Wed Jul 13 21:59:08 2022] GET /identity/v3/groups/061e2f5095f2431198cdcc9162dee432 => generated 226 bytes in 340 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 21:59:08.635608 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a895f0ac-4156-4014-b9bb-1a068743a0a8 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:08.686856 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d593679-4d30-4d4d-a903-8a59debc063a None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:08.713949 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:59:08.725905 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06e4ccb-73ea-45ae-b793-a8bf36b3b1c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:08.727150 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06e4ccb-73ea-45ae-b793-a8bf36b3b1c7 None admin] REQUEST_METHOD: `PUT` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:08.727719 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06e4ccb-73ea-45ae-b793-a8bf36b3b1c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:08.728207 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a06e4ccb-73ea-45ae-b793-a8bf36b3b1c7 None admin] PATH_INFO: `/v3/projects/0b902d068f594f05b4de544301553ded/groups/1c51f1a766f44a05b929ec468d2d908c/roles/44efba2579f342df8a0789c59d163ec3` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:08.731505 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06e4ccb-73ea-45ae-b793-a8bf36b3b1c7 None admin] RBAC: Authorizing `identity:create_grant(project_id=0b902d068f594f05b4de544301553ded, group_id=1c51f1a766f44a05b929ec468d2d908c, role_id=44efba2579f342df8a0789c59d163ec3)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:08.733297 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a06e4ccb-73ea-45ae-b793-a8bf36b3b1c7 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:08.764574 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 211/413] 10.222.0.52 () {58 vars in 1442 bytes} [Wed Jul 13 21:59:08 2022] PUT /identity/v3/projects/0b902d068f594f05b4de544301553ded/groups/1c51f1a766f44a05b929ec468d2d908c/roles/44efba2579f342df8a0789c59d163ec3 => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 13 21:59:08.774480 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a895f0ac-4156-4014-b9bb-1a068743a0a8 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:08.810919 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b18111d5-5d0a-4521-99e3-f1ae986ec8b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:08.811815 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18111d5-5d0a-4521-99e3-f1ae986ec8b6 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:08.812098 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18111d5-5d0a-4521-99e3-f1ae986ec8b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:08.812372 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b18111d5-5d0a-4521-99e3-f1ae986ec8b6 None admin] PATH_INFO: `/v3/projects/4896b13c732d49d09947c2baaf9dc04d` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:08.813663 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18111d5-5d0a-4521-99e3-f1ae986ec8b6 None admin] RBAC: Authorizing `identity:get_project(project_id=4896b13c732d49d09947c2baaf9dc04d)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:08.815100 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b18111d5-5d0a-4521-99e3-f1ae986ec8b6 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:08.817185 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 203/414] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Jul 13 21:59:08 2022] GET /identity/v3/projects/4896b13c732d49d09947c2baaf9dc04d => generated 302 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 21:59:08.823836 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06e4ccb-73ea-45ae-b793-a8bf36b3b1c7 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:08.963161 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:59:08.978707 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a06e4ccb-73ea-45ae-b793-a8bf36b3b1c7 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:09.018668 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-536d3fb9-a5e0-4273-9cee-e61d0fd5ddb9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:09.019635 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536d3fb9-a5e0-4273-9cee-e61d0fd5ddb9 None admin] REQUEST_METHOD: `PUT` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:09.019911 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536d3fb9-a5e0-4273-9cee-e61d0fd5ddb9 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:09.020182 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-536d3fb9-a5e0-4273-9cee-e61d0fd5ddb9 None admin] PATH_INFO: `/v3/projects/4896b13c732d49d09947c2baaf9dc04d/groups/061e2f5095f2431198cdcc9162dee432/roles/db84c6c897534af1a60b9ce62532f383` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:09.022477 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536d3fb9-a5e0-4273-9cee-e61d0fd5ddb9 None admin] RBAC: Authorizing `identity:create_grant(project_id=4896b13c732d49d09947c2baaf9dc04d, group_id=061e2f5095f2431198cdcc9162dee432, role_id=db84c6c897534af1a60b9ce62532f383)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:09.023961 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-536d3fb9-a5e0-4273-9cee-e61d0fd5ddb9 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:09.045969 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 212/415] 10.222.0.52 () {58 vars in 1442 bytes} [Wed Jul 13 21:59:08 2022] PUT /identity/v3/projects/4896b13c732d49d09947c2baaf9dc04d/groups/061e2f5095f2431198cdcc9162dee432/roles/db84c6c897534af1a60b9ce62532f383 => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 21:59:09.081018 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba00f2b-9a7f-44d9-a908-db7e4ff10e4f None None] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:09.081367 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba00f2b-9a7f-44d9-a908-db7e4ff10e4f None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:09.081592 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ba00f2b-9a7f-44d9-a908-db7e4ff10e4f None None] PATH_INFO: `/` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:09.083493 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 204/416] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jul 13 21:59:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 21:59:09.090123 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3503b74-de65-45a3-bef3-e34a9211cc07 None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:09.090472 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3503b74-de65-45a3-bef3-e34a9211cc07 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:09.090715 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3503b74-de65-45a3-bef3-e34a9211cc07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:09.113108 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-a3503b74-de65-45a3-bef3-e34a9211cc07 None None] MFA Rules not processed for user `cc99c93da5074cf5a48bc2aa9a92c5e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:59:09.114354 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-a3503b74-de65-45a3-bef3-e34a9211cc07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:59:09.216277 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 213/417] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jul 13 21:59:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 13 21:59:09.248951 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba00f2b-9a7f-44d9-a908-db7e4ff10e4f None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:09.364231 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ba00f2b-9a7f-44d9-a908-db7e4ff10e4f None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:09.374352 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:59:09.410005 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd520de5-ee5c-45a1-801f-a2abfeb8f159 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:09.410861 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd520de5-ee5c-45a1-801f-a2abfeb8f159 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:09.411142 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd520de5-ee5c-45a1-801f-a2abfeb8f159 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:09.411415 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd520de5-ee5c-45a1-801f-a2abfeb8f159 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:09.425649 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd520de5-ee5c-45a1-801f-a2abfeb8f159 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 21:59:09.434118 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application [None req-dd520de5-ee5c-45a1-801f-a2abfeb8f159 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 13 21:59:09.434118 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 21:59:09.434118 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 21:59:09.434118 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 21:59:09.434118 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 21:59:09.434118 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 21:59:09.434118 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 21:59:09.434118 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 21:59:09.434118 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 21:59:09.434118 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 21:59:09.434118 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 21:59:09.434118 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 21:59:09.434118 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 21:59:09.434118 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 21:59:09.434118 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 21:59:09.434118 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 21:59:09.436260 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 21:59:09.436260 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 21:59:09.436260 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 21:59:09.436260 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 21:59:09.436260 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 21:59:09.436260 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 21:59:09.436260 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 21:59:09.436260 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 21:59:09.436260 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 21:59:09.436260 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 21:59:09.436260 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 21:59:09.436260 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application with Lock( Jul 13 21:59:09.436260 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 21:59:09.436260 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._enter() Jul 13 21:59:09.436260 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 21:59:09.436260 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 21:59:09.438161 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 21:59:09.438161 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.creator() Jul 13 21:59:09.438161 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 21:59:09.438161 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application created_value = creator( Jul 13 21:59:09.438161 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 21:59:09.438161 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 21:59:09.438161 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 21:59:09.438161 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 21:59:09.438161 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 21:59:09.438161 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 21:59:09.438161 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 13 21:59:09.438161 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Jul 13 21:59:09.438161 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 205/418] 10.222.0.52 () {56 vars in 1114 bytes} [Wed Jul 13 21:59:09 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 21:59:09.443594 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3503b74-de65-45a3-bef3-e34a9211cc07 None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:09.554960 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:59:09.593148 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3503b74-de65-45a3-bef3-e34a9211cc07 None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:09.633925 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ccfa6f44-e0b0-4e97-9f89-adb9b67dd68e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:09.634655 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfa6f44-e0b0-4e97-9f89-adb9b67dd68e None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:09.634933 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfa6f44-e0b0-4e97-9f89-adb9b67dd68e None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:09.635204 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ccfa6f44-e0b0-4e97-9f89-adb9b67dd68e None admin] PATH_INFO: `/v3/roles` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:09.635969 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccfa6f44-e0b0-4e97-9f89-adb9b67dd68e None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 21:59:09.636269 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccfa6f44-e0b0-4e97-9f89-adb9b67dd68e None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:09.637657 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ccfa6f44-e0b0-4e97-9f89-adb9b67dd68e None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:09.650708 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 214/419] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Jul 13 21:59:09 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 203 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 21:59:09.652498 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd520de5-ee5c-45a1-801f-a2abfeb8f159 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:09.710441 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7930f6f-7497-4fb5-bfdd-cb62ebf4f4a0 None None] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:09.711119 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7930f6f-7497-4fb5-bfdd-cb62ebf4f4a0 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:09.711119 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7930f6f-7497-4fb5-bfdd-cb62ebf4f4a0 None None] PATH_INFO: `/` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:09.712514 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 215/420] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jul 13 21:59:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 13 21:59:09.727903 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfe142a-ce7a-403b-9b9c-55f7f95f0ac6 None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:09.728211 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfe142a-ce7a-403b-9b9c-55f7f95f0ac6 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:09.728503 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fdfe142a-ce7a-403b-9b9c-55f7f95f0ac6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:09.750042 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-fdfe142a-ce7a-403b-9b9c-55f7f95f0ac6 None None] MFA Rules not processed for user `cc99c93da5074cf5a48bc2aa9a92c5e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:59:09.755350 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-fdfe142a-ce7a-403b-9b9c-55f7f95f0ac6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:59:09.783963 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd520de5-ee5c-45a1-801f-a2abfeb8f159 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:09.850577 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d326e4-7bd2-4504-a1f6-e4188ff4b195 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:09.851151 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d326e4-7bd2-4504-a1f6-e4188ff4b195 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:09.851434 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d326e4-7bd2-4504-a1f6-e4188ff4b195 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:09.851710 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24d326e4-7bd2-4504-a1f6-e4188ff4b195 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:09.898379 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 216/421] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jul 13 21:59:09 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 13 21:59:09.911084 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:59:09.940410 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdfe142a-ce7a-403b-9b9c-55f7f95f0ac6 None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:09.964831 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d326e4-7bd2-4504-a1f6-e4188ff4b195 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:09.966407 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24d326e4-7bd2-4504-a1f6-e4188ff4b195 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:10.058335 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:59:10.129123 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fdfe142a-ce7a-403b-9b9c-55f7f95f0ac6 None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:10.142282 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 206/422] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Jul 13 21:59:09 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 490 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 13 21:59:10.148481 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d326e4-7bd2-4504-a1f6-e4188ff4b195 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:10.193162 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f735ae85-8094-47cb-80c1-bbe450a64f18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:10.195072 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f735ae85-8094-47cb-80c1-bbe450a64f18 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:10.195072 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f735ae85-8094-47cb-80c1-bbe450a64f18 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:10.195267 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f735ae85-8094-47cb-80c1-bbe450a64f18 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:10.211420 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f735ae85-8094-47cb-80c1-bbe450a64f18 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 21:59:10.220422 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application [None req-f735ae85-8094-47cb-80c1-bbe450a64f18 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 13 21:59:10.220422 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 21:59:10.220422 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 21:59:10.220422 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 21:59:10.220422 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 21:59:10.220422 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 21:59:10.220422 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 21:59:10.220422 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 21:59:10.220422 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 21:59:10.220422 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 21:59:10.220422 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 21:59:10.220422 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 21:59:10.220422 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 21:59:10.220422 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 21:59:10.220422 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 21:59:10.220422 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 21:59:10.221327 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 21:59:10.221327 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 21:59:10.221327 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 21:59:10.221327 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 21:59:10.221327 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 21:59:10.221327 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 21:59:10.221327 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 21:59:10.221327 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 21:59:10.221327 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 21:59:10.221327 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 21:59:10.221327 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 21:59:10.221327 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application with Lock( Jul 13 21:59:10.221327 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 21:59:10.221327 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._enter() Jul 13 21:59:10.221327 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 21:59:10.221327 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 21:59:10.222237 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 21:59:10.222237 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.creator() Jul 13 21:59:10.222237 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 21:59:10.222237 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application created_value = creator( Jul 13 21:59:10.222237 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 21:59:10.222237 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 21:59:10.222237 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 21:59:10.222237 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 21:59:10.222237 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 21:59:10.222237 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 21:59:10.222237 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 13 21:59:10.222237 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Jul 13 21:59:10.222237 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 217/423] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Jul 13 21:59:09 2022] GET /identity/v3/roles/member => generated 84 bytes in 282 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 13 21:59:10.223408 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfd40f6-9e02-42f0-9734-1ca024843354 None None] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:10.223678 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfd40f6-9e02-42f0-9734-1ca024843354 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:10.223952 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dfd40f6-9e02-42f0-9734-1ca024843354 None None] PATH_INFO: `/` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:10.225346 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 218/424] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jul 13 21:59:10 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 21:59:10.231093 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfd40f6-9e02-42f0-9734-1ca024843354 None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:10.236846 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:59:10.270525 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24d326e4-7bd2-4504-a1f6-e4188ff4b195 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:10.308195 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-898adf24-65ee-4409-9f5d-443867f53634 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:10.309084 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898adf24-65ee-4409-9f5d-443867f53634 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:10.309372 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898adf24-65ee-4409-9f5d-443867f53634 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:10.309648 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-898adf24-65ee-4409-9f5d-443867f53634 None admin] PATH_INFO: `/v3/projects/97aa9f72c7a54c9daa7e48288b4aab0f` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:10.310881 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-898adf24-65ee-4409-9f5d-443867f53634 None admin] RBAC: Authorizing `identity:get_project(project_id=97aa9f72c7a54c9daa7e48288b4aab0f)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:10.312349 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-898adf24-65ee-4409-9f5d-443867f53634 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:10.314384 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 207/425] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Jul 13 21:59:10 2022] GET /identity/v3/projects/97aa9f72c7a54c9daa7e48288b4aab0f => generated 298 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 21:59:10.315584 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9b8e7e-351c-41b3-b9f4-b950b525dabb None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:10.315872 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9b8e7e-351c-41b3-b9f4-b950b525dabb None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:10.316146 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d9b8e7e-351c-41b3-b9f4-b950b525dabb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:10.333669 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-3d9b8e7e-351c-41b3-b9f4-b950b525dabb None None] MFA Rules not processed for user `cc99c93da5074cf5a48bc2aa9a92c5e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:59:10.335013 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-3d9b8e7e-351c-41b3-b9f4-b950b525dabb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:59:10.348481 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dfd40f6-9e02-42f0-9734-1ca024843354 None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:10.375745 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:59:10.388769 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75faedec-972e-4028-9837-74cdb0665b03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:10.389930 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75faedec-972e-4028-9837-74cdb0665b03 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:10.390284 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75faedec-972e-4028-9837-74cdb0665b03 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:10.390656 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75faedec-972e-4028-9837-74cdb0665b03 None admin] PATH_INFO: `/v3/roles` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:10.391486 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75faedec-972e-4028-9837-74cdb0665b03 None admin] RBAC: Adding query filter params (name=member) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 21:59:10.391861 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75faedec-972e-4028-9837-74cdb0665b03 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:10.394107 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75faedec-972e-4028-9837-74cdb0665b03 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:10.407255 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 219/426] 10.222.0.52 () {56 vars in 1101 bytes} [Wed Jul 13 21:59:10 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 21:59:10.407868 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 208/427] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jul 13 21:59:10 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 13 21:59:10.412599 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9b8e7e-351c-41b3-b9f4-b950b525dabb None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:10.415639 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75faedec-972e-4028-9837-74cdb0665b03 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:10.495610 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:59:10.535622 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d9b8e7e-351c-41b3-b9f4-b950b525dabb None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:10.566677 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75faedec-972e-4028-9837-74cdb0665b03 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:10.579655 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282d7806-0e48-474f-92d1-d1e0da925732 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:10.580512 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282d7806-0e48-474f-92d1-d1e0da925732 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:10.580983 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282d7806-0e48-474f-92d1-d1e0da925732 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:10.581462 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-282d7806-0e48-474f-92d1-d1e0da925732 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:10.609828 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be169cd-bcde-455c-8683-69ea4a2c3f57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:10.610965 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be169cd-bcde-455c-8683-69ea4a2c3f57 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:10.611288 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be169cd-bcde-455c-8683-69ea4a2c3f57 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:10.611570 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6be169cd-bcde-455c-8683-69ea4a2c3f57 None admin] PATH_INFO: `/v3/groups/061e2f5095f2431198cdcc9162dee432` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:10.613317 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be169cd-bcde-455c-8683-69ea4a2c3f57 None admin] RBAC: Authorizing `identity:get_group(group_id=061e2f5095f2431198cdcc9162dee432)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:10.614995 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6be169cd-bcde-455c-8683-69ea4a2c3f57 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:10.622509 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 220/428] 10.222.0.52 () {56 vars in 1180 bytes} [Wed Jul 13 21:59:10 2022] GET /identity/v3/groups/061e2f5095f2431198cdcc9162dee432 => generated 226 bytes in 210 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 21:59:10.622509 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be169cd-bcde-455c-8683-69ea4a2c3f57 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:10.664223 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282d7806-0e48-474f-92d1-d1e0da925732 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:10.665693 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-282d7806-0e48-474f-92d1-d1e0da925732 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:10.684136 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:59:10.735999 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6be169cd-bcde-455c-8683-69ea4a2c3f57 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:10.753476 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:59:10.776136 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d43758-d36e-4baf-976a-40d38e530355 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:10.777264 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d43758-d36e-4baf-976a-40d38e530355 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:10.777754 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d43758-d36e-4baf-976a-40d38e530355 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:10.778297 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d43758-d36e-4baf-976a-40d38e530355 None admin] PATH_INFO: `/v3/roles/member` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:10.790008 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84d43758-d36e-4baf-976a-40d38e530355 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 21:59:10.801074 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 209/429] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Jul 13 21:59:10 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 384 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 13 21:59:10.801074 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282d7806-0e48-474f-92d1-d1e0da925732 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:10.804493 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application [None req-84d43758-d36e-4baf-976a-40d38e530355 None admin] Could not find role: member.: keystone.exception.RoleNotFound: Could not find role: member. Jul 13 21:59:10.804493 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 21:59:10.804493 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 21:59:10.804493 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 21:59:10.804493 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 21:59:10.804493 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 21:59:10.804493 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 21:59:10.804493 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 21:59:10.804493 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 21:59:10.804493 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 21:59:10.804493 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 21:59:10.804493 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 21:59:10.804493 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 21:59:10.804493 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 21:59:10.804493 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 21:59:10.804493 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 21:59:10.805393 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 21:59:10.805393 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 21:59:10.805393 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 21:59:10.805393 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 21:59:10.805393 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 21:59:10.805393 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 21:59:10.805393 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 21:59:10.805393 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 21:59:10.805393 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 21:59:10.805393 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 21:59:10.805393 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 21:59:10.805393 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application with Lock( Jul 13 21:59:10.805393 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 21:59:10.805393 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._enter() Jul 13 21:59:10.805393 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 21:59:10.805393 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 21:59:10.806209 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 21:59:10.806209 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.creator() Jul 13 21:59:10.806209 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 21:59:10.806209 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application created_value = creator( Jul 13 21:59:10.806209 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 21:59:10.806209 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 21:59:10.806209 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 21:59:10.806209 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 21:59:10.806209 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 21:59:10.806209 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 21:59:10.806209 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: member. Jul 13 21:59:10.806209 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Jul 13 21:59:10.808800 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 221/430] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Jul 13 21:59:10 2022] GET /identity/v3/roles/member => generated 84 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 21:59:10.809905 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d43758-d36e-4baf-976a-40d38e530355 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:10.926680 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84d43758-d36e-4baf-976a-40d38e530355 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:10.934799 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-282d7806-0e48-474f-92d1-d1e0da925732 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:10.943726 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:59:10.968808 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30205318-c3cc-455c-a8ee-dc5ac7b05a09 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:10.970177 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30205318-c3cc-455c-a8ee-dc5ac7b05a09 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:10.971305 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30205318-c3cc-455c-a8ee-dc5ac7b05a09 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:10.971761 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30205318-c3cc-455c-a8ee-dc5ac7b05a09 None admin] PATH_INFO: `/v3/projects/97aa9f72c7a54c9daa7e48288b4aab0f` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:10.973405 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30205318-c3cc-455c-a8ee-dc5ac7b05a09 None admin] RBAC: Authorizing `identity:get_project(project_id=97aa9f72c7a54c9daa7e48288b4aab0f)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:10.975103 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30205318-c3cc-455c-a8ee-dc5ac7b05a09 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:10.981557 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9c9579-d7b6-464a-9fdf-3dba9debcfb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:10.982398 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9c9579-d7b6-464a-9fdf-3dba9debcfb2 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:10.982629 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 222/431] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Jul 13 21:59:10 2022] GET /identity/v3/projects/97aa9f72c7a54c9daa7e48288b4aab0f => generated 298 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 21:59:10.982727 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9c9579-d7b6-464a-9fdf-3dba9debcfb2 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:10.983033 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad9c9579-d7b6-464a-9fdf-3dba9debcfb2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:10.983949 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9c9579-d7b6-464a-9fdf-3dba9debcfb2 None admin] RBAC: Adding query filter params (group.id=061e2f5095f2431198cdcc9162dee432, role.id=db84c6c897534af1a60b9ce62532f383, scope.project.id=97aa9f72c7a54c9daa7e48288b4aab0f) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 21:59:10.984153 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30205318-c3cc-455c-a8ee-dc5ac7b05a09 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:10.984323 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9c9579-d7b6-464a-9fdf-3dba9debcfb2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:10.985974 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad9c9579-d7b6-464a-9fdf-3dba9debcfb2 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:10.993622 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 210/432] 10.222.0.52 () {56 vars in 1376 bytes} [Wed Jul 13 21:59:10 2022] GET /identity/v3/role_assignments?group.id=061e2f5095f2431198cdcc9162dee432&scope.project.id=97aa9f72c7a54c9daa7e48288b4aab0f&role.id=db84c6c897534af1a60b9ce62532f383 => generated 611 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 21:59:10.994823 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9c9579-d7b6-464a-9fdf-3dba9debcfb2 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:11.035216 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:59:11.112621 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30205318-c3cc-455c-a8ee-dc5ac7b05a09 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:11.121003 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad9c9579-d7b6-464a-9fdf-3dba9debcfb2 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:11.164624 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a4848f-32e7-48db-b5a6-e4942e26fd96 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:11.165825 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a4848f-32e7-48db-b5a6-e4942e26fd96 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:11.166547 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a4848f-32e7-48db-b5a6-e4942e26fd96 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:11.166898 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87a4848f-32e7-48db-b5a6-e4942e26fd96 None admin] PATH_INFO: `/v3/groups/061e2f5095f2431198cdcc9162dee432` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:11.168350 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a4848f-32e7-48db-b5a6-e4942e26fd96 None admin] RBAC: Authorizing `identity:get_group(group_id=061e2f5095f2431198cdcc9162dee432)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:11.169988 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87a4848f-32e7-48db-b5a6-e4942e26fd96 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:11.172116 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 211/433] 10.222.0.52 () {56 vars in 1180 bytes} [Wed Jul 13 21:59:10 2022] GET /identity/v3/groups/061e2f5095f2431198cdcc9162dee432 => generated 226 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 21:59:11.178531 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a4848f-32e7-48db-b5a6-e4942e26fd96 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:11.179417 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-514d401d-bc76-4f7b-acc9-24657c99e857 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:11.180208 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514d401d-bc76-4f7b-acc9-24657c99e857 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:11.180589 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514d401d-bc76-4f7b-acc9-24657c99e857 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:11.181003 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-514d401d-bc76-4f7b-acc9-24657c99e857 None admin] PATH_INFO: `/v3/roles` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:11.182000 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-514d401d-bc76-4f7b-acc9-24657c99e857 None admin] RBAC: Adding query filter params (name=member) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 21:59:11.182589 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-514d401d-bc76-4f7b-acc9-24657c99e857 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:11.184385 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-514d401d-bc76-4f7b-acc9-24657c99e857 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:11.209016 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 223/434] 10.222.0.52 () {56 vars in 1101 bytes} [Wed Jul 13 21:59:10 2022] GET /identity/v3/roles?name=member => generated 341 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 21:59:11.218196 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c57aae9-cef1-4440-9e5b-68231cd817d4 None None] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:11.218196 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c57aae9-cef1-4440-9e5b-68231cd817d4 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:11.218471 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c57aae9-cef1-4440-9e5b-68231cd817d4 None None] PATH_INFO: `/` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:11.220182 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 224/435] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jul 13 21:59:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 13 21:59:11.220938 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c57aae9-cef1-4440-9e5b-68231cd817d4 None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:11.291488 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:59:11.300033 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:59:11.305571 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87a4848f-32e7-48db-b5a6-e4942e26fd96 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:11.345765 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11853276-fb70-4ebe-9147-a178a0a89a95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:11.346455 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11853276-fb70-4ebe-9147-a178a0a89a95 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:11.346765 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11853276-fb70-4ebe-9147-a178a0a89a95 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:11.347042 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11853276-fb70-4ebe-9147-a178a0a89a95 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:11.347901 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11853276-fb70-4ebe-9147-a178a0a89a95 None admin] RBAC: Adding query filter params (group.id=061e2f5095f2431198cdcc9162dee432, role.id=2a5e86976394498abb6310697cdb3f8b, scope.project.id=97aa9f72c7a54c9daa7e48288b4aab0f) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 21:59:11.348323 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11853276-fb70-4ebe-9147-a178a0a89a95 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:11.349873 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11853276-fb70-4ebe-9147-a178a0a89a95 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:11.356585 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 212/436] 10.222.0.52 () {56 vars in 1376 bytes} [Wed Jul 13 21:59:11 2022] GET /identity/v3/role_assignments?group.id=061e2f5095f2431198cdcc9162dee432&scope.project.id=97aa9f72c7a54c9daa7e48288b4aab0f&role.id=2a5e86976394498abb6310697cdb3f8b => generated 611 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 21:59:11.358524 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c8eaf6-fe84-464d-881d-ad6373240616 None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:11.358524 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c8eaf6-fe84-464d-881d-ad6373240616 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:11.358524 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96c8eaf6-fe84-464d-881d-ad6373240616 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:11.371981 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c57aae9-cef1-4440-9e5b-68231cd817d4 None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:11.376906 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-96c8eaf6-fe84-464d-881d-ad6373240616 None None] MFA Rules not processed for user `cc99c93da5074cf5a48bc2aa9a92c5e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:59:11.378140 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-96c8eaf6-fe84-464d-881d-ad6373240616 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:59:11.413299 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab95ea26-b95b-42d1-8a7f-25234613e5a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:11.413966 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab95ea26-b95b-42d1-8a7f-25234613e5a1 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:11.414249 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab95ea26-b95b-42d1-8a7f-25234613e5a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:11.414547 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab95ea26-b95b-42d1-8a7f-25234613e5a1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:11.472348 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 213/437] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jul 13 21:59:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 13 21:59:11.473645 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1fabe5-a580-42e6-b919-55eb89ce642f None None] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:11.473932 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1fabe5-a580-42e6-b919-55eb89ce642f None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:11.474208 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b1fabe5-a580-42e6-b919-55eb89ce642f None None] PATH_INFO: `/` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:11.476793 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 214/438] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jul 13 21:59:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 21:59:11.482582 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d559690-5d9f-4ba0-924b-0bf345c116b0 None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:11.482582 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d559690-5d9f-4ba0-924b-0bf345c116b0 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:11.482582 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d559690-5d9f-4ba0-924b-0bf345c116b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:11.503528 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-8d559690-5d9f-4ba0-924b-0bf345c116b0 None None] MFA Rules not processed for user `cc99c93da5074cf5a48bc2aa9a92c5e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:59:11.504857 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab95ea26-b95b-42d1-8a7f-25234613e5a1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:11.504992 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-8d559690-5d9f-4ba0-924b-0bf345c116b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:59:11.506356 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab95ea26-b95b-42d1-8a7f-25234613e5a1 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:11.562598 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:59:11.585549 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 215/439] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jul 13 21:59:11 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 13 21:59:11.586413 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d559690-5d9f-4ba0-924b-0bf345c116b0 None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:11.615835 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:59:11.635211 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 225/440] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Jul 13 21:59:11 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 414 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 13 21:59:11.636457 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab95ea26-b95b-42d1-8a7f-25234613e5a1 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:11.708186 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d559690-5d9f-4ba0-924b-0bf345c116b0 None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:11.747124 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab95ea26-b95b-42d1-8a7f-25234613e5a1 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:11.749753 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b0c49f-5a7d-4e4c-9270-ad032f2d0637 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:11.751140 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b0c49f-5a7d-4e4c-9270-ad032f2d0637 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:11.751140 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b0c49f-5a7d-4e4c-9270-ad032f2d0637 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:11.751379 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45b0c49f-5a7d-4e4c-9270-ad032f2d0637 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:11.763926 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45b0c49f-5a7d-4e4c-9270-ad032f2d0637 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 21:59:11.772143 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application [None req-45b0c49f-5a7d-4e4c-9270-ad032f2d0637 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 13 21:59:11.772143 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 21:59:11.772143 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 21:59:11.772143 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 21:59:11.772143 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 21:59:11.772143 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 21:59:11.772143 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 21:59:11.772143 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 21:59:11.772143 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 21:59:11.772143 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 21:59:11.772143 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 21:59:11.772143 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 21:59:11.772143 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 21:59:11.772143 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 21:59:11.772143 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 21:59:11.772143 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 21:59:11.773061 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 21:59:11.773061 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 21:59:11.773061 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 21:59:11.773061 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 21:59:11.773061 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 21:59:11.773061 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 21:59:11.773061 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 21:59:11.773061 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 21:59:11.773061 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 21:59:11.773061 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 21:59:11.773061 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 21:59:11.773061 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application with Lock( Jul 13 21:59:11.773061 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 21:59:11.773061 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._enter() Jul 13 21:59:11.773061 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 21:59:11.773061 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 21:59:11.773894 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 21:59:11.773894 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.creator() Jul 13 21:59:11.773894 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 21:59:11.773894 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application created_value = creator( Jul 13 21:59:11.773894 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 21:59:11.773894 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 21:59:11.773894 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 21:59:11.773894 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 21:59:11.773894 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 21:59:11.773894 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 21:59:11.773894 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 13 21:59:11.773894 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Jul 13 21:59:11.773894 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 216/441] 10.222.0.52 () {56 vars in 1096 bytes} [Wed Jul 13 21:59:11 2022] GET /identity/v3/roles/admin => generated 83 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 21:59:11.773894 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b0c49f-5a7d-4e4c-9270-ad032f2d0637 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:11.790772 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7705df2-6d7a-4e2a-af08-1e373a7214ab None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:11.791605 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7705df2-6d7a-4e2a-af08-1e373a7214ab None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:11.791886 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7705df2-6d7a-4e2a-af08-1e373a7214ab None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:11.792163 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7705df2-6d7a-4e2a-af08-1e373a7214ab None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:11.803021 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7705df2-6d7a-4e2a-af08-1e373a7214ab None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 21:59:11.811366 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application [None req-f7705df2-6d7a-4e2a-af08-1e373a7214ab None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 13 21:59:11.811366 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 21:59:11.811366 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 21:59:11.811366 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 21:59:11.811366 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 21:59:11.811366 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 21:59:11.811366 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 21:59:11.811366 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 21:59:11.811366 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 21:59:11.811366 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 21:59:11.811366 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 21:59:11.811366 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 21:59:11.811366 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 21:59:11.811366 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 21:59:11.811366 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 21:59:11.811366 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 21:59:11.812278 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 21:59:11.812278 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 21:59:11.812278 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 21:59:11.812278 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 21:59:11.812278 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 21:59:11.812278 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 21:59:11.812278 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 21:59:11.812278 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 21:59:11.812278 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 21:59:11.812278 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 21:59:11.812278 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 21:59:11.812278 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application with Lock( Jul 13 21:59:11.812278 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 21:59:11.812278 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._enter() Jul 13 21:59:11.812278 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 21:59:11.812278 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 21:59:11.813493 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 21:59:11.813493 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.creator() Jul 13 21:59:11.813493 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 21:59:11.813493 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application created_value = creator( Jul 13 21:59:11.813493 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 21:59:11.813493 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 21:59:11.813493 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 21:59:11.813493 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 21:59:11.813493 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 21:59:11.813493 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 21:59:11.813493 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 13 21:59:11.813493 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Jul 13 21:59:11.813493 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 226/442] 10.222.0.52 () {56 vars in 1114 bytes} [Wed Jul 13 21:59:11 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 21:59:11.813493 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7705df2-6d7a-4e2a-af08-1e373a7214ab None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:11.817727 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:59:11.882201 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:59:11.904627 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45b0c49f-5a7d-4e4c-9270-ad032f2d0637 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:11.930162 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7705df2-6d7a-4e2a-af08-1e373a7214ab None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:11.941021 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f7ecc1-1a56-4d2f-a205-725f453e7e35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:11.941895 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f7ecc1-1a56-4d2f-a205-725f453e7e35 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:11.942175 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f7ecc1-1a56-4d2f-a205-725f453e7e35 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:11.942449 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71f7ecc1-1a56-4d2f-a205-725f453e7e35 None admin] PATH_INFO: `/v3/projects/4896b13c732d49d09947c2baaf9dc04d` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:11.943684 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f7ecc1-1a56-4d2f-a205-725f453e7e35 None admin] RBAC: Authorizing `identity:get_project(project_id=4896b13c732d49d09947c2baaf9dc04d)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:11.945158 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71f7ecc1-1a56-4d2f-a205-725f453e7e35 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:11.946906 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 217/443] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Jul 13 21:59:11 2022] GET /identity/v3/projects/4896b13c732d49d09947c2baaf9dc04d => generated 302 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 21:59:11.947857 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f7ecc1-1a56-4d2f-a205-725f453e7e35 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:11.967347 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22b9913-1e32-444e-899f-0dc3ece014f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:11.968017 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22b9913-1e32-444e-899f-0dc3ece014f7 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:11.968297 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22b9913-1e32-444e-899f-0dc3ece014f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:11.968569 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e22b9913-1e32-444e-899f-0dc3ece014f7 None admin] PATH_INFO: `/v3/roles` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:11.969318 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22b9913-1e32-444e-899f-0dc3ece014f7 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 21:59:11.969625 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22b9913-1e32-444e-899f-0dc3ece014f7 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:11.971041 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e22b9913-1e32-444e-899f-0dc3ece014f7 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:11.979103 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 227/444] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Jul 13 21:59:11 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 21:59:11.980022 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22b9913-1e32-444e-899f-0dc3ece014f7 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:12.042242 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:59:12.049839 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71f7ecc1-1a56-4d2f-a205-725f453e7e35 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:12.076597 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e22b9913-1e32-444e-899f-0dc3ece014f7 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:12.082847 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3723b56-35bc-4a0d-8c7a-2b3c2a96215f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:12.083604 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3723b56-35bc-4a0d-8c7a-2b3c2a96215f None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:12.083888 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3723b56-35bc-4a0d-8c7a-2b3c2a96215f None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:12.084164 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3723b56-35bc-4a0d-8c7a-2b3c2a96215f None admin] PATH_INFO: `/v3/roles` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:12.084922 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3723b56-35bc-4a0d-8c7a-2b3c2a96215f None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 21:59:12.085228 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3723b56-35bc-4a0d-8c7a-2b3c2a96215f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:12.086637 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3723b56-35bc-4a0d-8c7a-2b3c2a96215f None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:12.094424 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 218/445] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Jul 13 21:59:11 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 21:59:12.095226 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3723b56-35bc-4a0d-8c7a-2b3c2a96215f None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:12.109805 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f83afd-9553-4c79-bc65-0e468b5dba77 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:12.110911 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f83afd-9553-4c79-bc65-0e468b5dba77 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:12.111216 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f83afd-9553-4c79-bc65-0e468b5dba77 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:12.111488 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5f83afd-9553-4c79-bc65-0e468b5dba77 None admin] PATH_INFO: `/v3/groups/061e2f5095f2431198cdcc9162dee432` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:12.112694 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f83afd-9553-4c79-bc65-0e468b5dba77 None admin] RBAC: Authorizing `identity:get_group(group_id=061e2f5095f2431198cdcc9162dee432)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:12.113476 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:59:12.114135 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5f83afd-9553-4c79-bc65-0e468b5dba77 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:12.115950 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 228/446] 10.222.0.52 () {56 vars in 1180 bytes} [Wed Jul 13 21:59:11 2022] GET /identity/v3/groups/061e2f5095f2431198cdcc9162dee432 => generated 226 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 21:59:12.116852 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f83afd-9553-4c79-bc65-0e468b5dba77 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:12.199951 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3723b56-35bc-4a0d-8c7a-2b3c2a96215f None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:12.216174 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5f83afd-9553-4c79-bc65-0e468b5dba77 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:12.233433 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8b82a1-638e-464a-b8d6-1d760813aa30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:12.234090 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8b82a1-638e-464a-b8d6-1d760813aa30 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:12.234372 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8b82a1-638e-464a-b8d6-1d760813aa30 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:12.234685 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb8b82a1-638e-464a-b8d6-1d760813aa30 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:12.249138 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c746d64-0351-47d2-805e-0fd4bd3b02bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:12.249791 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c746d64-0351-47d2-805e-0fd4bd3b02bf None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:12.250078 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c746d64-0351-47d2-805e-0fd4bd3b02bf None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:12.250352 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c746d64-0351-47d2-805e-0fd4bd3b02bf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:12.253244 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:59:12.302157 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8b82a1-638e-464a-b8d6-1d760813aa30 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:12.303594 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb8b82a1-638e-464a-b8d6-1d760813aa30 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:12.316799 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:59:12.319533 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c746d64-0351-47d2-805e-0fd4bd3b02bf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:12.320982 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2c746d64-0351-47d2-805e-0fd4bd3b02bf None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:12.400564 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 219/447] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Jul 13 21:59:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 306 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 13 21:59:12.401489 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8b82a1-638e-464a-b8d6-1d760813aa30 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:12.424879 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 229/448] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Jul 13 21:59:12 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 13 21:59:12.425715 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c746d64-0351-47d2-805e-0fd4bd3b02bf None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:12.462084 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:59:12.498993 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb8b82a1-638e-464a-b8d6-1d760813aa30 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:12.512300 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:59:12.527940 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c746d64-0351-47d2-805e-0fd4bd3b02bf None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:12.534153 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84187f5-5465-4fe4-b908-b1cbcaf9366c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:12.534837 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84187f5-5465-4fe4-b908-b1cbcaf9366c None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:12.535119 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84187f5-5465-4fe4-b908-b1cbcaf9366c None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:12.535406 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d84187f5-5465-4fe4-b908-b1cbcaf9366c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:12.536218 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d84187f5-5465-4fe4-b908-b1cbcaf9366c None admin] RBAC: Adding query filter params (group.id=061e2f5095f2431198cdcc9162dee432, role.id=2a5e86976394498abb6310697cdb3f8b, scope.project.id=4896b13c732d49d09947c2baaf9dc04d) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 21:59:12.536524 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d84187f5-5465-4fe4-b908-b1cbcaf9366c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:12.537967 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d84187f5-5465-4fe4-b908-b1cbcaf9366c None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:12.544183 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 220/449] 10.222.0.52 () {56 vars in 1376 bytes} [Wed Jul 13 21:59:12 2022] GET /identity/v3/role_assignments?group.id=061e2f5095f2431198cdcc9162dee432&scope.project.id=4896b13c732d49d09947c2baaf9dc04d&role.id=2a5e86976394498abb6310697cdb3f8b => generated 611 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 21:59:12.544962 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84187f5-5465-4fe4-b908-b1cbcaf9366c None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:12.567825 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45e280f-ee83-432b-ab9d-558094e95b70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:12.568659 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45e280f-ee83-432b-ab9d-558094e95b70 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:12.568932 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45e280f-ee83-432b-ab9d-558094e95b70 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:12.569202 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45e280f-ee83-432b-ab9d-558094e95b70 None admin] PATH_INFO: `/v3/projects/0b902d068f594f05b4de544301553ded` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:12.570519 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45e280f-ee83-432b-ab9d-558094e95b70 None admin] RBAC: Authorizing `identity:get_project(project_id=0b902d068f594f05b4de544301553ded)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:12.572042 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f45e280f-ee83-432b-ab9d-558094e95b70 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:12.574079 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 230/450] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Jul 13 21:59:12 2022] GET /identity/v3/projects/0b902d068f594f05b4de544301553ded => generated 344 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 21:59:12.580016 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45e280f-ee83-432b-ab9d-558094e95b70 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:12.665849 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d84187f5-5465-4fe4-b908-b1cbcaf9366c None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:12.693339 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45e280f-ee83-432b-ab9d-558094e95b70 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:12.695055 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:59:12.708552 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb16dd2-71b3-49cb-83fa-dab55bfc7836 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:12.709913 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb16dd2-71b3-49cb-83fa-dab55bfc7836 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:12.710101 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb16dd2-71b3-49cb-83fa-dab55bfc7836 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:12.710274 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffb16dd2-71b3-49cb-83fa-dab55bfc7836 None admin] PATH_INFO: `/v3/projects/4896b13c732d49d09947c2baaf9dc04d` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:12.712357 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb16dd2-71b3-49cb-83fa-dab55bfc7836 None admin] RBAC: Authorizing `identity:get_project(project_id=4896b13c732d49d09947c2baaf9dc04d)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:12.713801 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffb16dd2-71b3-49cb-83fa-dab55bfc7836 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:12.715700 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 221/451] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Jul 13 21:59:12 2022] GET /identity/v3/projects/4896b13c732d49d09947c2baaf9dc04d => generated 302 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 21:59:12.722680 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb16dd2-71b3-49cb-83fa-dab55bfc7836 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:12.735160 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af02e281-442e-44e0-8f7b-640a70dc51bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:12.736030 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af02e281-442e-44e0-8f7b-640a70dc51bd None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:12.736322 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af02e281-442e-44e0-8f7b-640a70dc51bd None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:12.736581 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af02e281-442e-44e0-8f7b-640a70dc51bd None admin] PATH_INFO: `/v3/groups/1c51f1a766f44a05b929ec468d2d908c` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:12.737744 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af02e281-442e-44e0-8f7b-640a70dc51bd None admin] RBAC: Authorizing `identity:get_group(group_id=1c51f1a766f44a05b929ec468d2d908c)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:12.739218 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af02e281-442e-44e0-8f7b-640a70dc51bd None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:12.741146 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 231/452] 10.222.0.52 () {56 vars in 1180 bytes} [Wed Jul 13 21:59:12 2022] GET /identity/v3/groups/1c51f1a766f44a05b929ec468d2d908c => generated 229 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 21:59:12.746704 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af02e281-442e-44e0-8f7b-640a70dc51bd None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:12.776315 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:59:12.848188 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffb16dd2-71b3-49cb-83fa-dab55bfc7836 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:12.867757 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af02e281-442e-44e0-8f7b-640a70dc51bd None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:12.893424 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f983a6f3-8062-4eef-ac41-0e1c88e5da61 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:12.894206 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f983a6f3-8062-4eef-ac41-0e1c88e5da61 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:12.894506 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f983a6f3-8062-4eef-ac41-0e1c88e5da61 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:12.894799 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f983a6f3-8062-4eef-ac41-0e1c88e5da61 None admin] PATH_INFO: `/v3/groups/061e2f5095f2431198cdcc9162dee432` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:12.896009 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f983a6f3-8062-4eef-ac41-0e1c88e5da61 None admin] RBAC: Authorizing `identity:get_group(group_id=061e2f5095f2431198cdcc9162dee432)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:12.897469 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f983a6f3-8062-4eef-ac41-0e1c88e5da61 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:12.899165 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 222/453] 10.222.0.52 () {56 vars in 1180 bytes} [Wed Jul 13 21:59:12 2022] GET /identity/v3/groups/061e2f5095f2431198cdcc9162dee432 => generated 226 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 21:59:12.905042 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f983a6f3-8062-4eef-ac41-0e1c88e5da61 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:12.912815 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6194056-72e4-4c7c-af93-f96ccd39f0f6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:12.913505 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6194056-72e4-4c7c-af93-f96ccd39f0f6 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:12.913786 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6194056-72e4-4c7c-af93-f96ccd39f0f6 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:12.914050 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6194056-72e4-4c7c-af93-f96ccd39f0f6 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:12.915151 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6194056-72e4-4c7c-af93-f96ccd39f0f6 None admin] RBAC: Adding query filter params (group.id=1c51f1a766f44a05b929ec468d2d908c, role.id=44efba2579f342df8a0789c59d163ec3, scope.project.id=0b902d068f594f05b4de544301553ded) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 21:59:12.915463 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6194056-72e4-4c7c-af93-f96ccd39f0f6 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:12.917284 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6194056-72e4-4c7c-af93-f96ccd39f0f6 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:12.923733 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 232/454] 10.222.0.52 () {56 vars in 1376 bytes} [Wed Jul 13 21:59:12 2022] GET /identity/v3/role_assignments?group.id=1c51f1a766f44a05b929ec468d2d908c&scope.project.id=0b902d068f594f05b4de544301553ded&role.id=44efba2579f342df8a0789c59d163ec3 => generated 611 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 21:59:13.014032 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f983a6f3-8062-4eef-ac41-0e1c88e5da61 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:13.028698 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:59:13.052425 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857527e8-c8a1-4d91-ae62-01883d576c52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:13.053094 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857527e8-c8a1-4d91-ae62-01883d576c52 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:13.053371 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857527e8-c8a1-4d91-ae62-01883d576c52 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:13.053641 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857527e8-c8a1-4d91-ae62-01883d576c52 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:13.054518 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-857527e8-c8a1-4d91-ae62-01883d576c52 None admin] RBAC: Adding query filter params (group.id=061e2f5095f2431198cdcc9162dee432, role.id=db84c6c897534af1a60b9ce62532f383, scope.project.id=4896b13c732d49d09947c2baaf9dc04d) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 21:59:13.054827 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-857527e8-c8a1-4d91-ae62-01883d576c52 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:13.056281 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-857527e8-c8a1-4d91-ae62-01883d576c52 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:13.063163 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 223/455] 10.222.0.52 () {56 vars in 1376 bytes} [Wed Jul 13 21:59:12 2022] GET /identity/v3/role_assignments?group.id=061e2f5095f2431198cdcc9162dee432&scope.project.id=4896b13c732d49d09947c2baaf9dc04d&role.id=db84c6c897534af1a60b9ce62532f383 => generated 611 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 21:59:15.570991 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12c33c3-ef2d-4b4a-839b-076015e9445d None None] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:15.570991 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12c33c3-ef2d-4b4a-839b-076015e9445d None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:15.570991 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f12c33c3-ef2d-4b4a-839b-076015e9445d None None] PATH_INFO: `/` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:15.572035 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 233/456] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jul 13 21:59:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 21:59:15.578413 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6b40f2-6ac6-4f3c-a313-84b887f4172a None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:15.579343 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6b40f2-6ac6-4f3c-a313-84b887f4172a None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:15.579817 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6b40f2-6ac6-4f3c-a313-84b887f4172a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:15.610773 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-7a6b40f2-6ac6-4f3c-a313-84b887f4172a None None] MFA Rules not processed for user `cc99c93da5074cf5a48bc2aa9a92c5e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:59:15.612287 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-7a6b40f2-6ac6-4f3c-a313-84b887f4172a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:59:15.707823 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 224/457] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jul 13 21:59:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 13 21:59:15.753735 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12c33c3-ef2d-4b4a-839b-076015e9445d None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:15.795543 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:59:15.833941 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741c44a3-7b96-4645-8caf-9afa9fd83f38 None None] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:15.834570 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741c44a3-7b96-4645-8caf-9afa9fd83f38 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:15.835057 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-741c44a3-7b96-4645-8caf-9afa9fd83f38 None None] PATH_INFO: `/` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:15.837863 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 225/458] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jul 13 21:59:15 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 13 21:59:15.842104 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7173ea06-a790-4a18-81e5-787c66ce1720 None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:15.842644 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7173ea06-a790-4a18-81e5-787c66ce1720 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:15.843117 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7173ea06-a790-4a18-81e5-787c66ce1720 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:15.876062 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-7173ea06-a790-4a18-81e5-787c66ce1720 None None] MFA Rules not processed for user `cc99c93da5074cf5a48bc2aa9a92c5e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:59:15.877276 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-7173ea06-a790-4a18-81e5-787c66ce1720 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:59:15.901363 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f12c33c3-ef2d-4b4a-839b-076015e9445d None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:15.966274 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0f2fe4-65b2-4b7b-8ed7-7cf836358de5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:15.967118 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0f2fe4-65b2-4b7b-8ed7-7cf836358de5 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:15.967401 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0f2fe4-65b2-4b7b-8ed7-7cf836358de5 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:15.967678 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0f2fe4-65b2-4b7b-8ed7-7cf836358de5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:15.995638 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef0f2fe4-65b2-4b7b-8ed7-7cf836358de5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:15.997126 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef0f2fe4-65b2-4b7b-8ed7-7cf836358de5 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:16.000384 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 226/459] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jul 13 21:59:15 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 159 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 13 21:59:16.020808 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application [None req-ef0f2fe4-65b2-4b7b-8ed7-7cf836358de5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 21:59:16.020808 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 21:59:16.020808 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 21:59:16.020808 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 21:59:16.020808 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 21:59:16.020808 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 21:59:16.020808 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 21:59:16.020808 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 21:59:16.020808 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 21:59:16.020808 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 21:59:16.020808 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 21:59:16.020808 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 21:59:16.020808 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 21:59:16.020808 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 21:59:16.020808 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 21:59:16.021920 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 21:59:16.021920 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 21:59:16.021920 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 21:59:16.021920 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 21:59:16.021920 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 21:59:16.021920 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 21:59:16.021920 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 21:59:16.021920 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 21:59:16.021920 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application with Lock( Jul 13 21:59:16.021920 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 21:59:16.021920 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._enter() Jul 13 21:59:16.021920 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 21:59:16.021920 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 21:59:16.021920 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 21:59:16.021920 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.creator() Jul 13 21:59:16.021920 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 21:59:16.021920 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application created_value = creator( Jul 13 21:59:16.023046 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 21:59:16.023046 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 21:59:16.023046 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 21:59:16.023046 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Jul 13 21:59:16.025055 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 234/460] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Jul 13 21:59:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 21:59:16.030835 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7173ea06-a790-4a18-81e5-787c66ce1720 None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:16.039571 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0f2fe4-65b2-4b7b-8ed7-7cf836358de5 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:16.094728 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:59:16.203850 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7173ea06-a790-4a18-81e5-787c66ce1720 None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:16.226947 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:59:16.253327 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0f2fe4-65b2-4b7b-8ed7-7cf836358de5 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:16.271119 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0136c5d-11f5-40ad-b486-956f841c46eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:16.272222 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0136c5d-11f5-40ad-b486-956f841c46eb None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:16.272823 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0136c5d-11f5-40ad-b486-956f841c46eb None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:16.273291 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0136c5d-11f5-40ad-b486-956f841c46eb None admin] PATH_INFO: `/v3/domains` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:16.274224 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0136c5d-11f5-40ad-b486-956f841c46eb None admin] RBAC: Adding query filter params (name=Default) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 21:59:16.274870 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0136c5d-11f5-40ad-b486-956f841c46eb None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:16.276636 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0136c5d-11f5-40ad-b486-956f841c46eb None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:16.289829 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 227/461] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Jul 13 21:59:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 259 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 21:59:16.290181 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-510178f1-1c44-49f5-8ee3-9d22795f19b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:16.291532 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510178f1-1c44-49f5-8ee3-9d22795f19b2 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:16.291864 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510178f1-1c44-49f5-8ee3-9d22795f19b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:16.292146 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-510178f1-1c44-49f5-8ee3-9d22795f19b2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:16.293453 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b22d2b-f4d7-4fd3-b0fc-d76dd5563fcb None None] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:16.293960 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b22d2b-f4d7-4fd3-b0fc-d76dd5563fcb None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:16.294658 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03b22d2b-f4d7-4fd3-b0fc-d76dd5563fcb None None] PATH_INFO: `/` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:16.296942 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 228/462] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jul 13 21:59:16 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 21:59:16.297889 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b22d2b-f4d7-4fd3-b0fc-d76dd5563fcb None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:16.304516 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510178f1-1c44-49f5-8ee3-9d22795f19b2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:16.306081 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-510178f1-1c44-49f5-8ee3-9d22795f19b2 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:16.326173 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application [None req-510178f1-1c44-49f5-8ee3-9d22795f19b2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 21:59:16.326173 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 21:59:16.326173 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 21:59:16.326173 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 21:59:16.326173 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 21:59:16.326173 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 21:59:16.326173 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 21:59:16.326173 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 21:59:16.326173 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 21:59:16.326173 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 21:59:16.326173 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 21:59:16.326173 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 21:59:16.326173 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 21:59:16.326173 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 21:59:16.326173 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 21:59:16.327142 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 21:59:16.327142 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 21:59:16.327142 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 21:59:16.327142 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 21:59:16.327142 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 21:59:16.327142 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 21:59:16.327142 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 21:59:16.327142 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 21:59:16.327142 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application with Lock( Jul 13 21:59:16.327142 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 21:59:16.327142 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._enter() Jul 13 21:59:16.327142 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 21:59:16.327142 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 21:59:16.327142 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 21:59:16.327142 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.creator() Jul 13 21:59:16.327142 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 21:59:16.327142 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application created_value = creator( Jul 13 21:59:16.328007 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 21:59:16.328007 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 21:59:16.328007 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 21:59:16.328007 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Jul 13 21:59:16.328007 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 235/463] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Jul 13 21:59:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 288 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 21:59:16.329472 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80392060-0833-4794-802a-8d9a38d0c0e7 None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:16.329819 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80392060-0833-4794-802a-8d9a38d0c0e7 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:16.330245 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80392060-0833-4794-802a-8d9a38d0c0e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:16.353116 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-80392060-0833-4794-802a-8d9a38d0c0e7 None None] MFA Rules not processed for user `cc99c93da5074cf5a48bc2aa9a92c5e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:59:16.355060 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-80392060-0833-4794-802a-8d9a38d0c0e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:59:16.416005 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:59:16.423957 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03b22d2b-f4d7-4fd3-b0fc-d76dd5563fcb None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:16.427132 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 236/464] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jul 13 21:59:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 13 21:59:16.428043 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80392060-0833-4794-802a-8d9a38d0c0e7 None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:16.460247 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-774ed074-eea2-4854-903e-e3cad2a26c99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:16.461200 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774ed074-eea2-4854-903e-e3cad2a26c99 None admin] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:16.461636 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774ed074-eea2-4854-903e-e3cad2a26c99 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:16.462090 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-774ed074-eea2-4854-903e-e3cad2a26c99 None admin] PATH_INFO: `/v3/users` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:16.463109 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-774ed074-eea2-4854-903e-e3cad2a26c99 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:16.464873 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-774ed074-eea2-4854-903e-e3cad2a26c99 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:16.487206 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 229/465] 10.222.0.52 () {60 vars in 1132 bytes} [Wed Jul 13 21:59:16 2022] POST /identity/v3/users => generated 246 bytes in 190 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 21:59:16.488636 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e493075-f0d9-4c73-8ef8-116afd34caeb None None] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:16.489057 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e493075-f0d9-4c73-8ef8-116afd34caeb None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:16.489445 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e493075-f0d9-4c73-8ef8-116afd34caeb None None] PATH_INFO: `/` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:16.491012 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 230/466] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jul 13 21:59:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 21:59:16.495767 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e493075-f0d9-4c73-8ef8-116afd34caeb None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:16.508561 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:59:16.546447 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80392060-0833-4794-802a-8d9a38d0c0e7 None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:16.590592 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82bf662e-25fc-4bb6-970e-e455760d63b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:16.591391 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bf662e-25fc-4bb6-970e-e455760d63b3 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:16.591624 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bf662e-25fc-4bb6-970e-e455760d63b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:16.591900 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82bf662e-25fc-4bb6-970e-e455760d63b3 None admin] PATH_INFO: `/v3/domains` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:16.592649 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82bf662e-25fc-4bb6-970e-e455760d63b3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 21:59:16.593007 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82bf662e-25fc-4bb6-970e-e455760d63b3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:16.594449 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82bf662e-25fc-4bb6-970e-e455760d63b3 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:16.615114 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 237/467] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Jul 13 21:59:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 21:59:16.616585 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a683d2-2fa3-4459-995e-66507fdea477 None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:16.616874 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a683d2-2fa3-4459-995e-66507fdea477 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:16.617147 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5a683d2-2fa3-4459-995e-66507fdea477 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:16.635515 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e493075-f0d9-4c73-8ef8-116afd34caeb None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:16.636289 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-c5a683d2-2fa3-4459-995e-66507fdea477 None None] MFA Rules not processed for user `cc99c93da5074cf5a48bc2aa9a92c5e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:59:16.637612 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-c5a683d2-2fa3-4459-995e-66507fdea477 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:59:16.672452 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86fb77ac-38a2-4e88-a2e0-af3be98dd583 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:16.673259 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fb77ac-38a2-4e88-a2e0-af3be98dd583 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:16.673536 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fb77ac-38a2-4e88-a2e0-af3be98dd583 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:16.673804 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86fb77ac-38a2-4e88-a2e0-af3be98dd583 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:16.686000 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86fb77ac-38a2-4e88-a2e0-af3be98dd583 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:16.687493 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-86fb77ac-38a2-4e88-a2e0-af3be98dd583 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:16.705240 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application [None req-86fb77ac-38a2-4e88-a2e0-af3be98dd583 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 21:59:16.705240 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 21:59:16.705240 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 21:59:16.705240 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 21:59:16.705240 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 21:59:16.705240 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 21:59:16.705240 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 21:59:16.705240 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 21:59:16.705240 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 21:59:16.705240 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 21:59:16.705240 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 21:59:16.705240 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 21:59:16.705240 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 21:59:16.705240 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 21:59:16.705240 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 21:59:16.706134 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 21:59:16.706134 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 21:59:16.706134 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 21:59:16.706134 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 21:59:16.706134 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 21:59:16.706134 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 21:59:16.706134 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 21:59:16.706134 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 21:59:16.706134 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application with Lock( Jul 13 21:59:16.706134 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 21:59:16.706134 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._enter() Jul 13 21:59:16.706134 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 21:59:16.706134 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 21:59:16.706134 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 21:59:16.706134 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.creator() Jul 13 21:59:16.706134 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 21:59:16.706134 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application created_value = creator( Jul 13 21:59:16.707041 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 21:59:16.707041 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 21:59:16.707041 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 21:59:16.707041 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Jul 13 21:59:16.707041 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 231/468] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Jul 13 21:59:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 21:59:16.707310 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86fb77ac-38a2-4e88-a2e0-af3be98dd583 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:16.714365 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:59:16.715389 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 238/469] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jul 13 21:59:16 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 13 21:59:16.716272 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a683d2-2fa3-4459-995e-66507fdea477 None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:16.786947 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:59:16.834897 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5a683d2-2fa3-4459-995e-66507fdea477 None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:16.854470 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-86fb77ac-38a2-4e88-a2e0-af3be98dd583 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:16.869823 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac54e99-9b27-4ec8-9a91-60bb96bd5389 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:16.870430 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac54e99-9b27-4ec8-9a91-60bb96bd5389 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:16.870744 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac54e99-9b27-4ec8-9a91-60bb96bd5389 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:16.871023 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ac54e99-9b27-4ec8-9a91-60bb96bd5389 None admin] PATH_INFO: `/v3/domains` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:16.871760 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac54e99-9b27-4ec8-9a91-60bb96bd5389 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 21:59:16.872067 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac54e99-9b27-4ec8-9a91-60bb96bd5389 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:16.873469 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3ac54e99-9b27-4ec8-9a91-60bb96bd5389 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:16.885940 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 239/470] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Jul 13 21:59:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 21:59:16.886822 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac54e99-9b27-4ec8-9a91-60bb96bd5389 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:16.904108 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7bab62f-184e-4ab3-b232-8ae779d63a22 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:16.905059 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bab62f-184e-4ab3-b232-8ae779d63a22 None admin] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:16.905336 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bab62f-184e-4ab3-b232-8ae779d63a22 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:16.905608 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7bab62f-184e-4ab3-b232-8ae779d63a22 None admin] PATH_INFO: `/v3/users` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:16.906609 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7bab62f-184e-4ab3-b232-8ae779d63a22 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:16.908820 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7bab62f-184e-4ab3-b232-8ae779d63a22 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:16.936290 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 232/471] 10.222.0.52 () {60 vars in 1132 bytes} [Wed Jul 13 21:59:16 2022] POST /identity/v3/users => generated 248 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 21:59:16.936924 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7bab62f-184e-4ab3-b232-8ae779d63a22 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:17.045277 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:59:17.060472 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ac54e99-9b27-4ec8-9a91-60bb96bd5389 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:17.081483 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:59:17.091734 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7bab62f-184e-4ab3-b232-8ae779d63a22 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:17.104827 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a5a356-e686-4d51-b5f5-71a6c2011262 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:17.106126 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a5a356-e686-4d51-b5f5-71a6c2011262 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:17.106742 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a5a356-e686-4d51-b5f5-71a6c2011262 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:17.107234 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a5a356-e686-4d51-b5f5-71a6c2011262 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:17.121456 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a5a356-e686-4d51-b5f5-71a6c2011262 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:17.124202 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a5a356-e686-4d51-b5f5-71a6c2011262 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:17.131320 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1301b883-a573-4f4f-a36c-66dc68d01f0f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:17.131591 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1301b883-a573-4f4f-a36c-66dc68d01f0f None admin] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:17.132158 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1301b883-a573-4f4f-a36c-66dc68d01f0f None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:17.132158 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1301b883-a573-4f4f-a36c-66dc68d01f0f None admin] PATH_INFO: `/v3/users` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:17.132845 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1301b883-a573-4f4f-a36c-66dc68d01f0f None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:17.134357 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1301b883-a573-4f4f-a36c-66dc68d01f0f None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:17.143223 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application [None req-89a5a356-e686-4d51-b5f5-71a6c2011262 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 21:59:17.143223 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 21:59:17.143223 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 21:59:17.143223 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 21:59:17.143223 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 21:59:17.143223 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 21:59:17.143223 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 21:59:17.143223 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 21:59:17.143223 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 21:59:17.143223 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 21:59:17.143223 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 21:59:17.143223 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 21:59:17.143223 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 21:59:17.143223 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 21:59:17.143223 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 21:59:17.144092 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 21:59:17.144092 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 21:59:17.144092 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 21:59:17.144092 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 21:59:17.144092 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 21:59:17.144092 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 21:59:17.144092 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 21:59:17.144092 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 21:59:17.144092 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application with Lock( Jul 13 21:59:17.144092 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 21:59:17.144092 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._enter() Jul 13 21:59:17.144092 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 21:59:17.144092 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 21:59:17.144092 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 21:59:17.144092 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.creator() Jul 13 21:59:17.144092 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 21:59:17.144092 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application created_value = creator( Jul 13 21:59:17.145093 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 21:59:17.145093 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 21:59:17.145093 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 21:59:17.145093 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Jul 13 21:59:17.145093 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 240/472] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Jul 13 21:59:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 258 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 21:59:17.146022 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f5bb45-7885-428a-8061-48040a225d9c None None] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:17.146813 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f5bb45-7885-428a-8061-48040a225d9c None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:17.146813 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9f5bb45-7885-428a-8061-48040a225d9c None None] PATH_INFO: `/` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:17.148131 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 241/473] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jul 13 21:59:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 21:59:17.153225 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f5bb45-7885-428a-8061-48040a225d9c None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:17.165807 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 233/474] 10.222.0.52 () {60 vars in 1132 bytes} [Wed Jul 13 21:59:16 2022] POST /identity/v3/users => generated 248 bytes in 229 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 21:59:17.167298 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc3fa68-ee9f-48df-9e3e-4496d9af2654 None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:17.167592 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc3fa68-ee9f-48df-9e3e-4496d9af2654 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:17.167889 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bc3fa68-ee9f-48df-9e3e-4496d9af2654 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:17.187921 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-9bc3fa68-ee9f-48df-9e3e-4496d9af2654 None None] MFA Rules not processed for user `cc99c93da5074cf5a48bc2aa9a92c5e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:59:17.189185 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-9bc3fa68-ee9f-48df-9e3e-4496d9af2654 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:59:17.273904 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9f5bb45-7885-428a-8061-48040a225d9c None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:17.289694 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 234/475] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jul 13 21:59:17 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 13 21:59:17.315806 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc3fa68-ee9f-48df-9e3e-4496d9af2654 None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:17.329980 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c884a13-2d4c-47ca-9621-39eb80679136 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:17.331024 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c884a13-2d4c-47ca-9621-39eb80679136 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:17.331475 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c884a13-2d4c-47ca-9621-39eb80679136 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:17.331891 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c884a13-2d4c-47ca-9621-39eb80679136 None admin] PATH_INFO: `/v3/domains` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:17.332799 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c884a13-2d4c-47ca-9621-39eb80679136 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 21:59:17.333267 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c884a13-2d4c-47ca-9621-39eb80679136 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:17.334910 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c884a13-2d4c-47ca-9621-39eb80679136 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:17.352763 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 242/476] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Jul 13 21:59:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 21:59:17.355069 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c884a13-2d4c-47ca-9621-39eb80679136 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:17.384077 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:59:17.389557 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:59:17.455207 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bc3fa68-ee9f-48df-9e3e-4496d9af2654 None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:17.473914 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c884a13-2d4c-47ca-9621-39eb80679136 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:17.492436 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08284584-2473-4b85-bcce-0704f1a9067e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:17.493116 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08284584-2473-4b85-bcce-0704f1a9067e None admin] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:17.493394 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08284584-2473-4b85-bcce-0704f1a9067e None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:17.493668 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08284584-2473-4b85-bcce-0704f1a9067e None admin] PATH_INFO: `/v3/roles` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:17.495380 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08284584-2473-4b85-bcce-0704f1a9067e None admin] RBAC: Authorizing `identity:create_role()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:17.496787 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08284584-2473-4b85-bcce-0704f1a9067e None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:17.505864 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.sql.core [None req-08284584-2473-4b85-bcce-0704f1a9067e None admin] Conflict role: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 13 21:59:17.505864 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jul 13 21:59:17.505864 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [parameters: {'id': 'ba2c1622d67148c89a9b85dcc00f58c4', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jul 13 21:59:17.505864 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80382) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 13 21:59:17.516217 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25e4d6f-a47b-4ad2-9de9-780fb5626d7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:17.517280 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25e4d6f-a47b-4ad2-9de9-780fb5626d7b None admin] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:17.517827 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25e4d6f-a47b-4ad2-9de9-780fb5626d7b None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:17.518319 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e25e4d6f-a47b-4ad2-9de9-780fb5626d7b None admin] PATH_INFO: `/v3/users` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:17.519266 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25e4d6f-a47b-4ad2-9de9-780fb5626d7b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:17.521098 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e25e4d6f-a47b-4ad2-9de9-780fb5626d7b None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:17.526207 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application [None req-08284584-2473-4b85-bcce-0704f1a9067e None admin] Conflict occurred attempting to store role - Duplicate entry found with name anotherrole.: keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jul 13 21:59:17.526207 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 21:59:17.526207 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 13 21:59:17.526207 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 13 21:59:17.526207 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 13 21:59:17.526207 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 13 21:59:17.526207 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 13 21:59:17.526207 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application result = self._query(query) Jul 13 21:59:17.526207 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 13 21:59:17.526207 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application conn.query(q) Jul 13 21:59:17.526207 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 13 21:59:17.526207 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 13 21:59:17.526207 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 13 21:59:17.526207 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application result.read() Jul 13 21:59:17.526207 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 13 21:59:17.527161 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 13 21:59:17.527161 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 13 21:59:17.527161 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 13 21:59:17.527161 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 13 21:59:17.527161 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 13 21:59:17.527161 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 13 21:59:17.527161 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 13 21:59:17.527161 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 13 21:59:17.527161 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Jul 13 21:59:17.527161 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 13 21:59:17.527161 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Jul 13 21:59:17.527161 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 21:59:17.527161 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 13 21:59:17.527161 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 13 21:59:17.527161 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 33, in create_role Jul 13 21:59:17.527161 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return ref.to_dict() Jul 13 21:59:17.527161 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 13 21:59:17.527161 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application next(self.gen) Jul 13 21:59:17.528167 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 13 21:59:17.528167 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application yield resource Jul 13 21:59:17.528167 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 13 21:59:17.528167 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application next(self.gen) Jul 13 21:59:17.528167 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 13 21:59:17.528167 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application self.session.rollback() Jul 13 21:59:17.528167 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 13 21:59:17.528167 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application self.force_reraise() Jul 13 21:59:17.528167 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 13 21:59:17.528167 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application raise self.value Jul 13 21:59:17.528167 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 13 21:59:17.528167 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 13 21:59:17.528167 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 13 21:59:17.528167 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application session.commit() Jul 13 21:59:17.528167 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 13 21:59:17.528167 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 13 21:59:17.528167 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 13 21:59:17.529070 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application self._prepare_impl() Jul 13 21:59:17.529070 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 13 21:59:17.529070 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application self.session.flush() Jul 13 21:59:17.529070 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 13 21:59:17.529070 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application self._flush(objects) Jul 13 21:59:17.529070 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 13 21:59:17.529070 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 13 21:59:17.529070 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 13 21:59:17.529070 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application compat.raise_( Jul 13 21:59:17.529070 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 13 21:59:17.529070 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application raise exception Jul 13 21:59:17.529070 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 13 21:59:17.529070 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application flush_context.execute() Jul 13 21:59:17.529070 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 13 21:59:17.529070 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application rec.execute(self) Jul 13 21:59:17.529070 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 13 21:59:17.529070 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 13 21:59:17.530010 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 13 21:59:17.530010 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 13 21:59:17.530010 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jul 13 21:59:17.530010 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application c = connection._execute_20( Jul 13 21:59:17.530010 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 13 21:59:17.530010 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 13 21:59:17.530010 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 13 21:59:17.530010 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 13 21:59:17.530010 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 13 21:59:17.530010 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 13 21:59:17.530010 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 13 21:59:17.530010 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 13 21:59:17.530010 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 13 21:59:17.530010 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 13 21:59:17.530010 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 13 21:59:17.530010 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application raise exception Jul 13 21:59:17.530880 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 13 21:59:17.530880 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 13 21:59:17.530880 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 13 21:59:17.530880 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 13 21:59:17.530880 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 13 21:59:17.530880 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application result = self._query(query) Jul 13 21:59:17.530880 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 13 21:59:17.530880 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application conn.query(q) Jul 13 21:59:17.530880 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 13 21:59:17.530880 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 13 21:59:17.530880 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 13 21:59:17.530880 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application result.read() Jul 13 21:59:17.530880 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 13 21:59:17.530880 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 13 21:59:17.530880 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 13 21:59:17.530880 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 13 21:59:17.531669 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 13 21:59:17.531669 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 13 21:59:17.531669 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 13 21:59:17.531669 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 13 21:59:17.531669 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'anotherrole-<>' for key 'role.ixu_role_name_domain_id'") Jul 13 21:59:17.531669 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application [SQL: INSERT INTO `role` (id, name, description, extra) VALUES (%(id)s, %(name)s, %(description)s, %(extra)s)] Jul 13 21:59:17.531669 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application [parameters: {'id': 'ba2c1622d67148c89a9b85dcc00f58c4', 'name': 'anotherrole', 'description': None, 'extra': '{}'}] Jul 13 21:59:17.531669 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 13 21:59:17.531669 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Jul 13 21:59:17.531669 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 13 21:59:17.531669 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Jul 13 21:59:17.531669 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 21:59:17.531669 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 21:59:17.531669 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 21:59:17.531669 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 21:59:17.531669 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 21:59:17.534208 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 21:59:17.534208 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 21:59:17.534208 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 21:59:17.534208 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 21:59:17.534208 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 21:59:17.534208 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 21:59:17.534208 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 115, in post Jul 13 21:59:17.534208 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application ref = PROVIDERS.role_api.create_role( Jul 13 21:59:17.534208 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 21:59:17.534208 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 21:59:17.534208 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1298, in create_role Jul 13 21:59:17.534208 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application ret = self.driver.create_role(role_id, role) Jul 13 21:59:17.534208 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 13 21:59:17.534208 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 13 21:59:17.534208 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store role - Duplicate entry found with name anotherrole. Jul 13 21:59:17.534208 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Jul 13 21:59:17.536359 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 235/477] 10.222.0.52 () {60 vars in 1131 bytes} [Wed Jul 13 21:59:17 2022] POST /identity/v3/roles => generated 144 bytes in 213 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Jul 13 21:59:17.536359 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08284584-2473-4b85-bcce-0704f1a9067e None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:17.566564 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 243/478] 10.222.0.52 () {60 vars in 1132 bytes} [Wed Jul 13 21:59:17 2022] POST /identity/v3/users => generated 249 bytes in 213 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 21:59:17.677858 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08284584-2473-4b85-bcce-0704f1a9067e None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:17.693921 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:59:17.726316 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d61a920c-19f5-472b-bf02-da6adaf4c645 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:17.727189 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61a920c-19f5-472b-bf02-da6adaf4c645 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:17.727470 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61a920c-19f5-472b-bf02-da6adaf4c645 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:17.727744 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d61a920c-19f5-472b-bf02-da6adaf4c645 None admin] PATH_INFO: `/v3/roles/anotherrole` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:17.743615 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d61a920c-19f5-472b-bf02-da6adaf4c645 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 21:59:17.752635 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application [None req-d61a920c-19f5-472b-bf02-da6adaf4c645 None admin] Could not find role: anotherrole.: keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 13 21:59:17.752635 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 21:59:17.752635 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 21:59:17.752635 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 21:59:17.752635 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 21:59:17.752635 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 21:59:17.752635 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 21:59:17.752635 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 21:59:17.752635 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 21:59:17.752635 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 21:59:17.752635 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 21:59:17.752635 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 21:59:17.752635 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 21:59:17.752635 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 21:59:17.752635 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 21:59:17.752635 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 21:59:17.753603 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 21:59:17.753603 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 21:59:17.753603 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 21:59:17.753603 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 21:59:17.753603 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 21:59:17.753603 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 21:59:17.753603 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 21:59:17.753603 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 21:59:17.753603 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 21:59:17.753603 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 21:59:17.753603 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 21:59:17.753603 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application with Lock( Jul 13 21:59:17.753603 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 21:59:17.753603 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._enter() Jul 13 21:59:17.753603 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 21:59:17.753603 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 21:59:17.754411 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 21:59:17.754411 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.creator() Jul 13 21:59:17.754411 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 21:59:17.754411 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application created_value = creator( Jul 13 21:59:17.754411 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 21:59:17.754411 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 21:59:17.754411 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 21:59:17.754411 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 21:59:17.754411 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 21:59:17.754411 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 21:59:17.754411 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: anotherrole. Jul 13 21:59:17.754411 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Jul 13 21:59:17.754411 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 236/479] 10.222.0.52 () {56 vars in 1114 bytes} [Wed Jul 13 21:59:17 2022] GET /identity/v3/roles/anotherrole => generated 89 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 21:59:17.759583 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25e4d6f-a47b-4ad2-9de9-780fb5626d7b None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:17.868014 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:59:17.884784 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e25e4d6f-a47b-4ad2-9de9-780fb5626d7b None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:17.928937 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9172fd38-5ee2-4974-9f8c-d92cac92aac9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:17.929557 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9172fd38-5ee2-4974-9f8c-d92cac92aac9 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:17.929842 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9172fd38-5ee2-4974-9f8c-d92cac92aac9 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:17.930115 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9172fd38-5ee2-4974-9f8c-d92cac92aac9 None admin] PATH_INFO: `/v3/roles` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:17.930930 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9172fd38-5ee2-4974-9f8c-d92cac92aac9 None admin] RBAC: Adding query filter params (name=anotherrole) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 21:59:17.931270 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9172fd38-5ee2-4974-9f8c-d92cac92aac9 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:17.932729 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9172fd38-5ee2-4974-9f8c-d92cac92aac9 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:17.944239 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 244/480] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Jul 13 21:59:17 2022] GET /identity/v3/roles?name=anotherrole => generated 334 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 21:59:18.866007 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed90016-7b88-43e3-9491-b642d183ddba None None] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:18.866537 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed90016-7b88-43e3-9491-b642d183ddba None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:18.866537 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed90016-7b88-43e3-9491-b642d183ddba None None] PATH_INFO: `/` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:18.868166 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 237/481] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jul 13 21:59:18 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 21:59:18.873493 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3737e438-aac7-4508-a8c5-e0f6db274f3b None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:18.873920 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3737e438-aac7-4508-a8c5-e0f6db274f3b None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:18.874089 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3737e438-aac7-4508-a8c5-e0f6db274f3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:18.899473 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-3737e438-aac7-4508-a8c5-e0f6db274f3b None None] MFA Rules not processed for user `cc99c93da5074cf5a48bc2aa9a92c5e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:59:18.901011 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-3737e438-aac7-4508-a8c5-e0f6db274f3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:59:18.986240 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 245/482] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jul 13 21:59:18 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 13 21:59:19.023558 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed90016-7b88-43e3-9491-b642d183ddba None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:19.163757 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed90016-7b88-43e3-9491-b642d183ddba None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:19.213647 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33774bc-5d6e-44d6-9500-56409b9fe997 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:19.214458 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33774bc-5d6e-44d6-9500-56409b9fe997 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:19.214764 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33774bc-5d6e-44d6-9500-56409b9fe997 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:19.215040 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c33774bc-5d6e-44d6-9500-56409b9fe997 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:19.231001 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c33774bc-5d6e-44d6-9500-56409b9fe997 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 21:59:19.238854 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application [None req-c33774bc-5d6e-44d6-9500-56409b9fe997 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 13 21:59:19.238854 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 21:59:19.238854 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 21:59:19.238854 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 21:59:19.238854 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 21:59:19.238854 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 21:59:19.238854 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 21:59:19.238854 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 21:59:19.238854 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 21:59:19.238854 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 21:59:19.238854 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 21:59:19.238854 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 21:59:19.238854 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 21:59:19.238854 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 21:59:19.238854 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 21:59:19.238854 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 21:59:19.239755 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 21:59:19.239755 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 21:59:19.239755 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 21:59:19.239755 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 21:59:19.239755 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 21:59:19.239755 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 21:59:19.239755 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 21:59:19.239755 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 21:59:19.239755 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 21:59:19.239755 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 21:59:19.239755 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 21:59:19.239755 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application with Lock( Jul 13 21:59:19.239755 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 21:59:19.239755 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._enter() Jul 13 21:59:19.239755 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 21:59:19.239755 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 21:59:19.240561 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 21:59:19.240561 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.creator() Jul 13 21:59:19.240561 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 21:59:19.240561 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application created_value = creator( Jul 13 21:59:19.240561 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 21:59:19.240561 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 21:59:19.240561 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 21:59:19.240561 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 21:59:19.240561 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 21:59:19.240561 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 21:59:19.240561 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 13 21:59:19.240561 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Jul 13 21:59:19.245270 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 238/483] 10.222.0.52 () {56 vars in 1102 bytes} [Wed Jul 13 21:59:19 2022] GET /identity/v3/roles/service => generated 85 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 21:59:19.246717 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3737e438-aac7-4508-a8c5-e0f6db274f3b None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:19.386143 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:59:19.414661 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3737e438-aac7-4508-a8c5-e0f6db274f3b None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:19.465570 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a45b27ca-3193-4834-a437-c1aac2d443de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:19.466330 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45b27ca-3193-4834-a437-c1aac2d443de None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:19.466634 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45b27ca-3193-4834-a437-c1aac2d443de None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:19.466907 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a45b27ca-3193-4834-a437-c1aac2d443de None admin] PATH_INFO: `/v3/roles` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:19.467663 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45b27ca-3193-4834-a437-c1aac2d443de None admin] RBAC: Adding query filter params (name=service) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 21:59:19.467965 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45b27ca-3193-4834-a437-c1aac2d443de None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:19.469350 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a45b27ca-3193-4834-a437-c1aac2d443de None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:19.482849 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 246/484] 10.222.0.52 () {56 vars in 1103 bytes} [Wed Jul 13 21:59:19 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 236 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 21:59:19.488326 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33774bc-5d6e-44d6-9500-56409b9fe997 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:19.506146 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4aa868-4eeb-474f-8f83-82d6c7be301e None None] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:19.506457 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4aa868-4eeb-474f-8f83-82d6c7be301e None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:19.506727 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4aa868-4eeb-474f-8f83-82d6c7be301e None None] PATH_INFO: `/` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:19.508349 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 247/485] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jul 13 21:59:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 13 21:59:19.520498 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17cad0b-29af-4f71-a813-871a76fce56e None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:19.520498 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17cad0b-29af-4f71-a813-871a76fce56e None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:19.520498 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17cad0b-29af-4f71-a813-871a76fce56e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:19.548208 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:59:19.555548 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-c17cad0b-29af-4f71-a813-871a76fce56e None None] MFA Rules not processed for user `cc99c93da5074cf5a48bc2aa9a92c5e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:59:19.557154 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-c17cad0b-29af-4f71-a813-871a76fce56e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:59:19.680450 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 248/486] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jul 13 21:59:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 162 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 13 21:59:19.697083 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c33774bc-5d6e-44d6-9500-56409b9fe997 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:19.729454 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17cad0b-29af-4f71-a813-871a76fce56e None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:19.741599 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b39bb34-dec4-456c-9ba5-ff74e037360a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:19.742643 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b39bb34-dec4-456c-9ba5-ff74e037360a None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:19.743109 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b39bb34-dec4-456c-9ba5-ff74e037360a None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:19.743572 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b39bb34-dec4-456c-9ba5-ff74e037360a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:19.769296 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b39bb34-dec4-456c-9ba5-ff74e037360a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:19.769296 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b39bb34-dec4-456c-9ba5-ff74e037360a None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:19.794851 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application [None req-9b39bb34-dec4-456c-9ba5-ff74e037360a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 21:59:19.794851 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 21:59:19.794851 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 21:59:19.794851 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 21:59:19.794851 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 21:59:19.794851 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 21:59:19.794851 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 21:59:19.794851 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 21:59:19.794851 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 21:59:19.794851 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 21:59:19.794851 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 21:59:19.794851 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 21:59:19.794851 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 21:59:19.794851 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 21:59:19.794851 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 21:59:19.795772 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 21:59:19.795772 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 21:59:19.795772 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 21:59:19.795772 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 21:59:19.795772 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 21:59:19.795772 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 21:59:19.795772 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 21:59:19.795772 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 21:59:19.795772 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application with Lock( Jul 13 21:59:19.795772 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 21:59:19.795772 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._enter() Jul 13 21:59:19.795772 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 21:59:19.795772 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 21:59:19.795772 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 21:59:19.795772 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.creator() Jul 13 21:59:19.795772 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 21:59:19.795772 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application created_value = creator( Jul 13 21:59:19.796653 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 21:59:19.796653 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 21:59:19.796653 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 21:59:19.796653 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Jul 13 21:59:19.798966 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 239/487] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Jul 13 21:59:19 2022] GET /identity/v3/domains/Default => generated 87 bytes in 311 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 21:59:19.800583 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e21abb-523f-46f6-a219-b97da3ad49d2 None None] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:19.801066 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e21abb-523f-46f6-a219-b97da3ad49d2 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:19.801542 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62e21abb-523f-46f6-a219-b97da3ad49d2 None None] PATH_INFO: `/` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:19.803308 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 240/488] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jul 13 21:59:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 21:59:19.809738 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d064843-a180-4e3f-8702-21e44f13a0de None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:19.810309 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d064843-a180-4e3f-8702-21e44f13a0de None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:19.810831 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d064843-a180-4e3f-8702-21e44f13a0de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:19.832291 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-1d064843-a180-4e3f-8702-21e44f13a0de None None] MFA Rules not processed for user `cc99c93da5074cf5a48bc2aa9a92c5e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:59:19.833834 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-1d064843-a180-4e3f-8702-21e44f13a0de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:59:19.842617 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:59:19.890304 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17cad0b-29af-4f71-a813-871a76fce56e None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:19.926522 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 241/489] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jul 13 21:59:19 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 13 21:59:19.928210 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d064843-a180-4e3f-8702-21e44f13a0de None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:19.943353 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24dd1729-87aa-4442-866e-3457cbb78b26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:19.944506 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24dd1729-87aa-4442-866e-3457cbb78b26 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:19.945165 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24dd1729-87aa-4442-866e-3457cbb78b26 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:19.946209 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24dd1729-87aa-4442-866e-3457cbb78b26 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:19.957931 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:59:19.966625 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24dd1729-87aa-4442-866e-3457cbb78b26 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 21:59:19.976979 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application [None req-24dd1729-87aa-4442-866e-3457cbb78b26 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 13 21:59:19.976979 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 21:59:19.976979 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 21:59:19.976979 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 21:59:19.976979 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 21:59:19.976979 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 21:59:19.976979 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 21:59:19.976979 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 21:59:19.976979 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 21:59:19.976979 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 21:59:19.976979 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 21:59:19.976979 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 21:59:19.976979 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 21:59:19.976979 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 21:59:19.976979 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 21:59:19.976979 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 21:59:19.977954 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 21:59:19.977954 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 21:59:19.977954 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 21:59:19.977954 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 21:59:19.977954 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 21:59:19.977954 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 21:59:19.977954 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 21:59:19.977954 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 21:59:19.977954 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 21:59:19.977954 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 21:59:19.977954 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 21:59:19.977954 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application with Lock( Jul 13 21:59:19.977954 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 21:59:19.977954 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._enter() Jul 13 21:59:19.977954 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 21:59:19.977954 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 21:59:19.987170 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 21:59:19.987170 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.creator() Jul 13 21:59:19.987170 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 21:59:19.987170 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application created_value = creator( Jul 13 21:59:19.987170 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 21:59:19.987170 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 21:59:19.987170 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 21:59:19.987170 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 21:59:19.987170 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 21:59:19.987170 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 21:59:19.987170 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 13 21:59:19.987170 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Jul 13 21:59:19.991450 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 249/490] 10.222.0.52 () {56 vars in 1102 bytes} [Wed Jul 13 21:59:19 2022] GET /identity/v3/roles/service => generated 85 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 21:59:19.992448 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24dd1729-87aa-4442-866e-3457cbb78b26 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:20.070819 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d064843-a180-4e3f-8702-21e44f13a0de None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:20.109866 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24dd1729-87aa-4442-866e-3457cbb78b26 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:20.115763 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4973a16-e1ff-4b9b-9df4-e60e02b2bacf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:20.116818 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4973a16-e1ff-4b9b-9df4-e60e02b2bacf None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:20.117302 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4973a16-e1ff-4b9b-9df4-e60e02b2bacf None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:20.117773 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4973a16-e1ff-4b9b-9df4-e60e02b2bacf None admin] PATH_INFO: `/v3/domains` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:20.119077 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4973a16-e1ff-4b9b-9df4-e60e02b2bacf None admin] RBAC: Adding query filter params (name=Default) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 21:59:20.120109 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4973a16-e1ff-4b9b-9df4-e60e02b2bacf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:20.121794 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4973a16-e1ff-4b9b-9df4-e60e02b2bacf None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:20.137038 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 242/491] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Jul 13 21:59:19 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 208 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 21:59:20.138249 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4973a16-e1ff-4b9b-9df4-e60e02b2bacf None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:20.146340 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:59:20.154706 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a038af1-ae65-44a0-af43-f23fc967d3df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:20.155622 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a038af1-ae65-44a0-af43-f23fc967d3df None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:20.155913 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a038af1-ae65-44a0-af43-f23fc967d3df None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:20.156195 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a038af1-ae65-44a0-af43-f23fc967d3df None admin] PATH_INFO: `/v3/roles/service` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:20.169121 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a038af1-ae65-44a0-af43-f23fc967d3df None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 21:59:20.177675 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application [None req-6a038af1-ae65-44a0-af43-f23fc967d3df None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 13 21:59:20.177675 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 21:59:20.177675 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 21:59:20.177675 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 21:59:20.177675 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 21:59:20.177675 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 21:59:20.177675 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 21:59:20.177675 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 21:59:20.177675 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 21:59:20.177675 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 21:59:20.177675 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 21:59:20.177675 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 21:59:20.177675 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 21:59:20.177675 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 21:59:20.177675 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 21:59:20.177675 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 21:59:20.178617 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 21:59:20.178617 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 21:59:20.178617 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 21:59:20.178617 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 21:59:20.178617 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 21:59:20.178617 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 21:59:20.178617 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 21:59:20.178617 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 21:59:20.178617 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 21:59:20.178617 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 21:59:20.178617 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 21:59:20.178617 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application with Lock( Jul 13 21:59:20.178617 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 21:59:20.178617 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._enter() Jul 13 21:59:20.178617 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 21:59:20.178617 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 21:59:20.179414 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 21:59:20.179414 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.creator() Jul 13 21:59:20.179414 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 21:59:20.179414 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application created_value = creator( Jul 13 21:59:20.179414 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 21:59:20.179414 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 21:59:20.179414 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 21:59:20.179414 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 21:59:20.179414 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 21:59:20.179414 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 21:59:20.179414 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 13 21:59:20.179414 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Jul 13 21:59:20.186008 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 250/492] 10.222.0.52 () {56 vars in 1102 bytes} [Wed Jul 13 21:59:19 2022] GET /identity/v3/roles/service => generated 85 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 21:59:20.187435 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a038af1-ae65-44a0-af43-f23fc967d3df None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:20.233164 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:59:20.278293 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4973a16-e1ff-4b9b-9df4-e60e02b2bacf None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:20.305458 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a038af1-ae65-44a0-af43-f23fc967d3df None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:20.317320 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3d841e-a95f-4f0a-9a5f-d88eadb98f14 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:20.318004 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3d841e-a95f-4f0a-9a5f-d88eadb98f14 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:20.318281 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3d841e-a95f-4f0a-9a5f-d88eadb98f14 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:20.318575 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc3d841e-a95f-4f0a-9a5f-d88eadb98f14 None admin] PATH_INFO: `/v3/roles` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:20.319328 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3d841e-a95f-4f0a-9a5f-d88eadb98f14 None admin] RBAC: Adding query filter params (name=service) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 21:59:20.319635 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3d841e-a95f-4f0a-9a5f-d88eadb98f14 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:20.321297 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc3d841e-a95f-4f0a-9a5f-d88eadb98f14 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:20.330717 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 243/493] 10.222.0.52 () {56 vars in 1103 bytes} [Wed Jul 13 21:59:20 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 21:59:20.331544 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3d841e-a95f-4f0a-9a5f-d88eadb98f14 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:20.354386 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ad37391-efe2-4042-a97f-0c8b2b486086 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:20.355236 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad37391-efe2-4042-a97f-0c8b2b486086 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:20.355517 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad37391-efe2-4042-a97f-0c8b2b486086 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:20.355792 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ad37391-efe2-4042-a97f-0c8b2b486086 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:20.365603 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad37391-efe2-4042-a97f-0c8b2b486086 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:20.367598 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ad37391-efe2-4042-a97f-0c8b2b486086 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:20.384168 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application [None req-9ad37391-efe2-4042-a97f-0c8b2b486086 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 13 21:59:20.384168 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 21:59:20.384168 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 21:59:20.384168 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 21:59:20.384168 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 21:59:20.384168 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 21:59:20.384168 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 21:59:20.384168 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 21:59:20.384168 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 21:59:20.384168 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 21:59:20.384168 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 21:59:20.384168 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 21:59:20.384168 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 13 21:59:20.384168 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 13 21:59:20.384168 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 13 21:59:20.384168 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 13 21:59:20.385390 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 21:59:20.385390 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 21:59:20.385390 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 13 21:59:20.385390 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 21:59:20.385390 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 13 21:59:20.385390 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 21:59:20.385390 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 21:59:20.385390 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 21:59:20.385390 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 21:59:20.385390 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 21:59:20.385390 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 21:59:20.385390 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application with Lock( Jul 13 21:59:20.385390 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 21:59:20.385390 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._enter() Jul 13 21:59:20.385390 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 21:59:20.385390 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 21:59:20.385390 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 21:59:20.385390 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.creator() Jul 13 21:59:20.391515 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 21:59:20.391515 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application created_value = creator( Jul 13 21:59:20.391515 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 13 21:59:20.391515 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 13 21:59:20.391515 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 13 21:59:20.391515 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 13 21:59:20.391515 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 13 21:59:20.391515 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 13 21:59:20.391515 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 13 21:59:20.391515 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Jul 13 21:59:20.391515 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 251/494] 10.222.0.52 () {56 vars in 1093 bytes} [Wed Jul 13 21:59:20 2022] GET /identity/v3/users/nova => generated 82 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 21:59:20.391515 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b22a01-3abb-403c-882b-a916e82ecf69 None None] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:20.391515 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b22a01-3abb-403c-882b-a916e82ecf69 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:20.392903 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b22a01-3abb-403c-882b-a916e82ecf69 None None] PATH_INFO: `/` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:20.392903 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 252/495] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jul 13 21:59:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 21:59:20.392903 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b22a01-3abb-403c-882b-a916e82ecf69 None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:20.452430 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:59:20.456180 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc3d841e-a95f-4f0a-9a5f-d88eadb98f14 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:20.491302 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0da36dff-0bb9-4b1d-8b84-9679e1a6f412 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:20.491938 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da36dff-0bb9-4b1d-8b84-9679e1a6f412 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:20.492228 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da36dff-0bb9-4b1d-8b84-9679e1a6f412 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:20.492500 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0da36dff-0bb9-4b1d-8b84-9679e1a6f412 None admin] PATH_INFO: `/v3/roles` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:20.493265 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da36dff-0bb9-4b1d-8b84-9679e1a6f412 None admin] RBAC: Adding query filter params (name=service) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 21:59:20.493571 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da36dff-0bb9-4b1d-8b84-9679e1a6f412 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:20.495397 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0da36dff-0bb9-4b1d-8b84-9679e1a6f412 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:20.497040 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:59:20.505840 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 244/496] 10.222.0.52 () {56 vars in 1103 bytes} [Wed Jul 13 21:59:20 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 21:59:20.507112 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0e6b53-b44e-499b-a32d-333d861e1a3e None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:20.507401 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0e6b53-b44e-499b-a32d-333d861e1a3e None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:20.507677 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0e6b53-b44e-499b-a32d-333d861e1a3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:20.514136 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b22a01-3abb-403c-882b-a916e82ecf69 None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:20.524574 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-7c0e6b53-b44e-499b-a32d-333d861e1a3e None None] MFA Rules not processed for user `cc99c93da5074cf5a48bc2aa9a92c5e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:59:20.525911 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-7c0e6b53-b44e-499b-a32d-333d861e1a3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:59:20.556809 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7878f6d-8f40-462b-8cbd-eab0557e8725 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:20.557781 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7878f6d-8f40-462b-8cbd-eab0557e8725 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:20.558057 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7878f6d-8f40-462b-8cbd-eab0557e8725 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:20.558662 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7878f6d-8f40-462b-8cbd-eab0557e8725 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:20.571517 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7878f6d-8f40-462b-8cbd-eab0557e8725 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:20.573792 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7878f6d-8f40-462b-8cbd-eab0557e8725 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:20.595866 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 245/497] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jul 13 21:59:20 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 13 21:59:20.596186 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application [None req-f7878f6d-8f40-462b-8cbd-eab0557e8725 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 21:59:20.596186 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 21:59:20.596186 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 21:59:20.596186 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 21:59:20.596186 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 21:59:20.596186 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 21:59:20.596186 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 21:59:20.596186 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 21:59:20.596186 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 21:59:20.596186 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 21:59:20.596186 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 21:59:20.596186 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 21:59:20.596186 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 21:59:20.596186 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 21:59:20.596186 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 21:59:20.596993 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 21:59:20.596993 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 21:59:20.596993 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 21:59:20.596993 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 21:59:20.596993 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 21:59:20.596993 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 21:59:20.596993 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 21:59:20.596993 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 21:59:20.596993 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application with Lock( Jul 13 21:59:20.596993 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 21:59:20.596993 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._enter() Jul 13 21:59:20.596993 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 21:59:20.596993 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 21:59:20.596993 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 21:59:20.596993 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.creator() Jul 13 21:59:20.596993 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 21:59:20.596993 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application created_value = creator( Jul 13 21:59:20.597839 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 21:59:20.597839 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 21:59:20.597839 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 21:59:20.597839 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Jul 13 21:59:20.599366 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0e6b53-b44e-499b-a32d-333d861e1a3e None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:20.602142 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 253/498] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Jul 13 21:59:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 21:59:20.603212 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7878f6d-8f40-462b-8cbd-eab0557e8725 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:20.706913 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7878f6d-8f40-462b-8cbd-eab0557e8725 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:20.708451 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:59:20.711666 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0e6b53-b44e-499b-a32d-333d861e1a3e None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:20.737744 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:59:20.747933 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7402d2-0069-4c08-b766-749b2d50c484 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:20.748122 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5380809c-3691-489a-b664-6e2208923e7a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:20.748540 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7402d2-0069-4c08-b766-749b2d50c484 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:20.748702 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5380809c-3691-489a-b664-6e2208923e7a None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:20.748808 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7402d2-0069-4c08-b766-749b2d50c484 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:20.748952 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5380809c-3691-489a-b664-6e2208923e7a None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:20.749096 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9e7402d2-0069-4c08-b766-749b2d50c484 None admin] PATH_INFO: `/v3/users` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:20.749237 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5380809c-3691-489a-b664-6e2208923e7a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:20.749938 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7402d2-0069-4c08-b766-749b2d50c484 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 21:59:20.750223 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7402d2-0069-4c08-b766-749b2d50c484 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:20.751723 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9e7402d2-0069-4c08-b766-749b2d50c484 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:20.760679 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5380809c-3691-489a-b664-6e2208923e7a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:20.762144 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5380809c-3691-489a-b664-6e2208923e7a None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:20.783584 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application [None req-5380809c-3691-489a-b664-6e2208923e7a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 21:59:20.783584 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 21:59:20.783584 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 21:59:20.783584 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 21:59:20.783584 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 21:59:20.783584 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 21:59:20.783584 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 21:59:20.783584 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 21:59:20.783584 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 21:59:20.783584 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 21:59:20.783584 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 21:59:20.783584 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 21:59:20.783584 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 21:59:20.783584 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 21:59:20.783584 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 21:59:20.784739 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 21:59:20.784739 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 21:59:20.784739 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 21:59:20.784739 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 21:59:20.784739 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 21:59:20.784739 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 21:59:20.784739 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 21:59:20.784739 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 21:59:20.784739 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application with Lock( Jul 13 21:59:20.784739 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 21:59:20.784739 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._enter() Jul 13 21:59:20.784739 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 21:59:20.784739 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 21:59:20.784739 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 21:59:20.784739 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.creator() Jul 13 21:59:20.784739 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 21:59:20.784739 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application created_value = creator( Jul 13 21:59:20.785616 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 21:59:20.785616 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 21:59:20.785616 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 21:59:20.785616 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Jul 13 21:59:20.786270 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 254/499] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Jul 13 21:59:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 21:59:20.787156 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5380809c-3691-489a-b664-6e2208923e7a None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:20.793536 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 246/500] 10.222.0.52 () {56 vars in 1133 bytes} [Wed Jul 13 21:59:20 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 21:59:20.794795 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7402d2-0069-4c08-b766-749b2d50c484 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:20.890237 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5380809c-3691-489a-b664-6e2208923e7a None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:20.904771 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9e7402d2-0069-4c08-b766-749b2d50c484 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:20.926535 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5525a914-cc8c-44de-a2c7-c7475a7316f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:20.927190 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5525a914-cc8c-44de-a2c7-c7475a7316f3 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:20.927472 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5525a914-cc8c-44de-a2c7-c7475a7316f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:20.927751 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5525a914-cc8c-44de-a2c7-c7475a7316f3 None admin] PATH_INFO: `/v3/domains` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:20.928485 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5525a914-cc8c-44de-a2c7-c7475a7316f3 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 21:59:20.928795 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5525a914-cc8c-44de-a2c7-c7475a7316f3 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:20.930185 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5525a914-cc8c-44de-a2c7-c7475a7316f3 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:20.941836 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181f4f04-0240-4532-90f5-58445ff640ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:20.942410 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 255/501] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Jul 13 21:59:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 21:59:20.943252 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5525a914-cc8c-44de-a2c7-c7475a7316f3 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:20.943625 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181f4f04-0240-4532-90f5-58445ff640ad None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:20.943909 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181f4f04-0240-4532-90f5-58445ff640ad None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:20.944182 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-181f4f04-0240-4532-90f5-58445ff640ad None admin] PATH_INFO: `/v3/roles/service` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:20.957935 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-181f4f04-0240-4532-90f5-58445ff640ad None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 21:59:20.963459 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:59:20.966379 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application [None req-181f4f04-0240-4532-90f5-58445ff640ad None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 13 21:59:20.966379 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 21:59:20.966379 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 21:59:20.966379 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 21:59:20.966379 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 21:59:20.966379 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 21:59:20.966379 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 21:59:20.966379 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 21:59:20.966379 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 21:59:20.966379 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 21:59:20.966379 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 21:59:20.966379 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 21:59:20.966379 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 21:59:20.966379 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 21:59:20.966379 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 21:59:20.966379 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 21:59:20.967355 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 21:59:20.967355 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 21:59:20.967355 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 21:59:20.967355 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 21:59:20.967355 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 21:59:20.967355 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 21:59:20.967355 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 21:59:20.967355 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 21:59:20.967355 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 21:59:20.967355 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 21:59:20.967355 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 21:59:20.967355 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application with Lock( Jul 13 21:59:20.967355 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 21:59:20.967355 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._enter() Jul 13 21:59:20.967355 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 21:59:20.967355 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 21:59:20.968173 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 21:59:20.968173 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.creator() Jul 13 21:59:20.968173 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 21:59:20.968173 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application created_value = creator( Jul 13 21:59:20.968173 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 21:59:20.968173 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 21:59:20.968173 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 21:59:20.968173 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 21:59:20.968173 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 21:59:20.968173 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 21:59:20.968173 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 13 21:59:20.968173 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Jul 13 21:59:20.968173 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 247/502] 10.222.0.52 () {56 vars in 1102 bytes} [Wed Jul 13 21:59:20 2022] GET /identity/v3/roles/service => generated 85 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 21:59:20.968836 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181f4f04-0240-4532-90f5-58445ff640ad None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:21.013866 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:59:21.051377 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5525a914-cc8c-44de-a2c7-c7475a7316f3 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:21.067497 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-181f4f04-0240-4532-90f5-58445ff640ad None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:21.085791 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1591f919-d426-4223-bfed-46f98a674499 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:21.086458 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1591f919-d426-4223-bfed-46f98a674499 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:21.086768 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1591f919-d426-4223-bfed-46f98a674499 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:21.087043 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1591f919-d426-4223-bfed-46f98a674499 None admin] PATH_INFO: `/v3/domains` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:21.087975 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1591f919-d426-4223-bfed-46f98a674499 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 21:59:21.087975 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1591f919-d426-4223-bfed-46f98a674499 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:21.089652 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1591f919-d426-4223-bfed-46f98a674499 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:21.100962 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 256/503] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Jul 13 21:59:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 21:59:21.101734 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1591f919-d426-4223-bfed-46f98a674499 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:21.111844 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a944d0d-21a1-46a6-9d82-cab96ad00997 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:21.112869 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a944d0d-21a1-46a6-9d82-cab96ad00997 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:21.113152 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a944d0d-21a1-46a6-9d82-cab96ad00997 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:21.113430 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a944d0d-21a1-46a6-9d82-cab96ad00997 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:21.177067 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:59:21.186384 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a944d0d-21a1-46a6-9d82-cab96ad00997 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:21.187887 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a944d0d-21a1-46a6-9d82-cab96ad00997 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:21.204179 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1591f919-d426-4223-bfed-46f98a674499 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:21.231468 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:59:21.237418 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e334d8f9-40f8-4663-ad1f-27f8efbe7759 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:21.238222 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e334d8f9-40f8-4663-ad1f-27f8efbe7759 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:21.238541 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e334d8f9-40f8-4663-ad1f-27f8efbe7759 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:21.238816 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e334d8f9-40f8-4663-ad1f-27f8efbe7759 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:21.246387 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e334d8f9-40f8-4663-ad1f-27f8efbe7759 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:21.247906 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e334d8f9-40f8-4663-ad1f-27f8efbe7759 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:21.262510 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application [None req-e334d8f9-40f8-4663-ad1f-27f8efbe7759 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 13 21:59:21.262510 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 21:59:21.262510 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 21:59:21.262510 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 21:59:21.262510 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 21:59:21.262510 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 21:59:21.262510 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 21:59:21.262510 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 21:59:21.262510 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 21:59:21.262510 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 21:59:21.262510 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 21:59:21.262510 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 21:59:21.262510 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 13 21:59:21.262510 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 13 21:59:21.262510 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 13 21:59:21.262510 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 13 21:59:21.263399 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 21:59:21.263399 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 21:59:21.263399 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 13 21:59:21.263399 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 21:59:21.263399 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 13 21:59:21.263399 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 21:59:21.263399 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 21:59:21.263399 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 21:59:21.263399 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 21:59:21.263399 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 21:59:21.263399 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 21:59:21.263399 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application with Lock( Jul 13 21:59:21.263399 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 21:59:21.263399 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._enter() Jul 13 21:59:21.263399 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 21:59:21.263399 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 21:59:21.263399 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 21:59:21.263399 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.creator() Jul 13 21:59:21.264350 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 21:59:21.264350 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application created_value = creator( Jul 13 21:59:21.264350 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 13 21:59:21.264350 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 13 21:59:21.264350 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 13 21:59:21.264350 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 13 21:59:21.264350 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 13 21:59:21.264350 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 13 21:59:21.264350 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 13 21:59:21.264350 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Jul 13 21:59:21.264350 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 257/504] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Jul 13 21:59:21 2022] GET /identity/v3/users/glance => generated 84 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 21:59:21.265286 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029636c7-6e33-4ae0-b2c0-e13a52a830b6 None None] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:21.265578 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029636c7-6e33-4ae0-b2c0-e13a52a830b6 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:21.265851 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-029636c7-6e33-4ae0-b2c0-e13a52a830b6 None None] PATH_INFO: `/` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:21.267160 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 258/505] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jul 13 21:59:21 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 21:59:21.267828 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029636c7-6e33-4ae0-b2c0-e13a52a830b6 None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:21.293616 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 248/506] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Jul 13 21:59:20 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 13 21:59:21.294538 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a944d0d-21a1-46a6-9d82-cab96ad00997 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:21.382028 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-029636c7-6e33-4ae0-b2c0-e13a52a830b6 None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:21.395209 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a944d0d-21a1-46a6-9d82-cab96ad00997 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:21.417243 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378f4ef6-22f9-4b68-85ee-a41dc3e3bfbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:21.417906 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378f4ef6-22f9-4b68-85ee-a41dc3e3bfbf None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:21.418208 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378f4ef6-22f9-4b68-85ee-a41dc3e3bfbf None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:21.418517 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-378f4ef6-22f9-4b68-85ee-a41dc3e3bfbf None admin] PATH_INFO: `/v3/roles` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:21.419341 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-378f4ef6-22f9-4b68-85ee-a41dc3e3bfbf None admin] RBAC: Adding query filter params (name=service) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 21:59:21.419652 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-378f4ef6-22f9-4b68-85ee-a41dc3e3bfbf None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:21.421037 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-378f4ef6-22f9-4b68-85ee-a41dc3e3bfbf None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:21.422421 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:59:21.428707 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40f7906b-0df9-46a1-83af-1d85a3bff040 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:21.429506 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f7906b-0df9-46a1-83af-1d85a3bff040 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:21.429792 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f7906b-0df9-46a1-83af-1d85a3bff040 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:21.430061 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40f7906b-0df9-46a1-83af-1d85a3bff040 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:21.437183 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 259/507] 10.222.0.52 () {56 vars in 1103 bytes} [Wed Jul 13 21:59:21 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 21:59:21.437183 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378f4ef6-22f9-4b68-85ee-a41dc3e3bfbf None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:21.438081 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f7906b-0df9-46a1-83af-1d85a3bff040 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:21.439587 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40f7906b-0df9-46a1-83af-1d85a3bff040 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:21.456130 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application [None req-40f7906b-0df9-46a1-83af-1d85a3bff040 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 13 21:59:21.456130 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 21:59:21.456130 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 21:59:21.456130 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 21:59:21.456130 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 21:59:21.456130 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 21:59:21.456130 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 21:59:21.456130 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 21:59:21.456130 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 21:59:21.456130 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 21:59:21.456130 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 21:59:21.456130 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 21:59:21.456130 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 13 21:59:21.456130 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 13 21:59:21.456130 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 13 21:59:21.456130 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 13 21:59:21.457247 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 21:59:21.457247 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 21:59:21.457247 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 13 21:59:21.457247 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 21:59:21.457247 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 13 21:59:21.457247 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 21:59:21.457247 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 21:59:21.457247 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 21:59:21.457247 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 21:59:21.457247 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 21:59:21.457247 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 21:59:21.457247 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application with Lock( Jul 13 21:59:21.457247 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 21:59:21.457247 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._enter() Jul 13 21:59:21.457247 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 21:59:21.457247 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 21:59:21.457247 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 21:59:21.457247 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.creator() Jul 13 21:59:21.458185 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 21:59:21.458185 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application created_value = creator( Jul 13 21:59:21.458185 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 13 21:59:21.458185 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 13 21:59:21.458185 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 13 21:59:21.458185 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 13 21:59:21.458185 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 13 21:59:21.458185 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 13 21:59:21.458185 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 13 21:59:21.458185 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Jul 13 21:59:21.458185 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 249/508] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Jul 13 21:59:21 2022] GET /identity/v3/users/cinder => generated 84 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 21:59:21.458798 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3532870a-726c-4885-b68c-4ac8923a8fe6 None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:21.459293 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3532870a-726c-4885-b68c-4ac8923a8fe6 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:21.459293 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3532870a-726c-4885-b68c-4ac8923a8fe6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:21.465426 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:59:21.477441 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-3532870a-726c-4885-b68c-4ac8923a8fe6 None None] MFA Rules not processed for user `cc99c93da5074cf5a48bc2aa9a92c5e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:59:21.478719 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-3532870a-726c-4885-b68c-4ac8923a8fe6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:59:21.544524 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 250/509] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jul 13 21:59:21 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 13 21:59:21.545282 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3532870a-726c-4885-b68c-4ac8923a8fe6 None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:21.545701 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-378f4ef6-22f9-4b68-85ee-a41dc3e3bfbf None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:21.586232 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea01ba5a-5fb4-484c-a63f-e08efd2a640b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:21.587026 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea01ba5a-5fb4-484c-a63f-e08efd2a640b None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:21.587304 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea01ba5a-5fb4-484c-a63f-e08efd2a640b None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:21.587764 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea01ba5a-5fb4-484c-a63f-e08efd2a640b None admin] PATH_INFO: `/v3/users` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:21.588802 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea01ba5a-5fb4-484c-a63f-e08efd2a640b None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 21:59:21.589109 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea01ba5a-5fb4-484c-a63f-e08efd2a640b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:21.590581 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea01ba5a-5fb4-484c-a63f-e08efd2a640b None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:21.626391 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 260/510] 10.222.0.52 () {56 vars in 1137 bytes} [Wed Jul 13 21:59:21 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 21:59:21.627261 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea01ba5a-5fb4-484c-a63f-e08efd2a640b None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:21.643542 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3532870a-726c-4885-b68c-4ac8923a8fe6 None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:21.678176 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1cc4c5-588b-400b-ac26-94c1d52d03da None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:21.679008 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1cc4c5-588b-400b-ac26-94c1d52d03da None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:21.679272 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1cc4c5-588b-400b-ac26-94c1d52d03da None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:21.679541 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c1cc4c5-588b-400b-ac26-94c1d52d03da None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:21.680563 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:59:21.681629 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:59:21.691568 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1cc4c5-588b-400b-ac26-94c1d52d03da None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:21.693279 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c1cc4c5-588b-400b-ac26-94c1d52d03da None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:21.713423 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application [None req-4c1cc4c5-588b-400b-ac26-94c1d52d03da None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 21:59:21.713423 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 21:59:21.713423 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 21:59:21.713423 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 21:59:21.713423 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 21:59:21.713423 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 21:59:21.713423 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 21:59:21.713423 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 21:59:21.713423 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 21:59:21.713423 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 21:59:21.713423 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 21:59:21.713423 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 21:59:21.713423 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 21:59:21.713423 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 21:59:21.713423 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 21:59:21.714449 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 21:59:21.714449 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 21:59:21.714449 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 21:59:21.714449 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 21:59:21.714449 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 21:59:21.714449 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 21:59:21.714449 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 21:59:21.714449 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 21:59:21.714449 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application with Lock( Jul 13 21:59:21.714449 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 21:59:21.714449 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._enter() Jul 13 21:59:21.714449 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 21:59:21.714449 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 21:59:21.714449 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 21:59:21.714449 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.creator() Jul 13 21:59:21.714449 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 21:59:21.714449 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application created_value = creator( Jul 13 21:59:21.715511 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 21:59:21.715511 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 21:59:21.715511 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 21:59:21.715511 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Jul 13 21:59:21.715511 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 251/511] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Jul 13 21:59:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 21:59:21.715511 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1cc4c5-588b-400b-ac26-94c1d52d03da None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:21.728911 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea01ba5a-5fb4-484c-a63f-e08efd2a640b None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:21.764367 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b47f36-be0e-4f6d-a7d8-d86f3fd4066e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:21.765163 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b47f36-be0e-4f6d-a7d8-d86f3fd4066e None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:21.765452 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b47f36-be0e-4f6d-a7d8-d86f3fd4066e None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:21.765716 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b47f36-be0e-4f6d-a7d8-d86f3fd4066e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:21.776414 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b47f36-be0e-4f6d-a7d8-d86f3fd4066e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:21.777921 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0b47f36-be0e-4f6d-a7d8-d86f3fd4066e None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:21.795017 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application [None req-d0b47f36-be0e-4f6d-a7d8-d86f3fd4066e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 21:59:21.795017 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 21:59:21.795017 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 21:59:21.795017 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 21:59:21.795017 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 21:59:21.795017 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 21:59:21.795017 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 21:59:21.795017 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 21:59:21.795017 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 21:59:21.795017 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 21:59:21.795017 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 21:59:21.795017 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 21:59:21.795017 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 21:59:21.795017 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 21:59:21.795017 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 21:59:21.795879 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 21:59:21.795879 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 21:59:21.795879 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 21:59:21.795879 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 21:59:21.795879 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 21:59:21.795879 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 21:59:21.795879 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 21:59:21.795879 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 21:59:21.795879 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application with Lock( Jul 13 21:59:21.795879 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 21:59:21.795879 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._enter() Jul 13 21:59:21.795879 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 21:59:21.795879 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 21:59:21.795879 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 21:59:21.795879 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.creator() Jul 13 21:59:21.795879 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 21:59:21.795879 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application created_value = creator( Jul 13 21:59:21.796724 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 21:59:21.796724 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 21:59:21.796724 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 21:59:21.796724 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Jul 13 21:59:21.796724 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 261/512] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Jul 13 21:59:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 21:59:21.796995 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b47f36-be0e-4f6d-a7d8-d86f3fd4066e None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:21.816017 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c1cc4c5-588b-400b-ac26-94c1d52d03da None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:21.850547 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e366f84b-8daf-45b6-901c-497621c4bad8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:21.851229 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e366f84b-8daf-45b6-901c-497621c4bad8 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:21.851491 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e366f84b-8daf-45b6-901c-497621c4bad8 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:21.851757 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e366f84b-8daf-45b6-901c-497621c4bad8 None admin] PATH_INFO: `/v3/users` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:21.852596 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e366f84b-8daf-45b6-901c-497621c4bad8 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 21:59:21.852900 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e366f84b-8daf-45b6-901c-497621c4bad8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:21.854578 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e366f84b-8daf-45b6-901c-497621c4bad8 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:21.871491 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 252/513] 10.222.0.52 () {56 vars in 1137 bytes} [Wed Jul 13 21:59:21 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 21:59:21.872307 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e366f84b-8daf-45b6-901c-497621c4bad8 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:21.899846 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0b47f36-be0e-4f6d-a7d8-d86f3fd4066e None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:21.906977 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:59:21.916790 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:59:21.936164 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db9f6d7-8a9a-4fd4-aad3-46deb11ac2fb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:21.936967 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db9f6d7-8a9a-4fd4-aad3-46deb11ac2fb None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:21.937249 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db9f6d7-8a9a-4fd4-aad3-46deb11ac2fb None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:21.937522 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db9f6d7-8a9a-4fd4-aad3-46deb11ac2fb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:21.947936 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db9f6d7-8a9a-4fd4-aad3-46deb11ac2fb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:21.949751 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db9f6d7-8a9a-4fd4-aad3-46deb11ac2fb None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:21.967426 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application [None req-9db9f6d7-8a9a-4fd4-aad3-46deb11ac2fb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 21:59:21.967426 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 21:59:21.967426 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 21:59:21.967426 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 21:59:21.967426 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 21:59:21.967426 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 21:59:21.967426 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 21:59:21.967426 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 21:59:21.967426 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 21:59:21.967426 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 21:59:21.967426 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 21:59:21.967426 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 21:59:21.967426 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 21:59:21.967426 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 21:59:21.967426 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 21:59:21.968440 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 21:59:21.968440 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 21:59:21.968440 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 21:59:21.968440 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 21:59:21.968440 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 21:59:21.968440 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 21:59:21.968440 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 21:59:21.968440 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 21:59:21.968440 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application with Lock( Jul 13 21:59:21.968440 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 21:59:21.968440 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._enter() Jul 13 21:59:21.968440 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 21:59:21.968440 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 21:59:21.968440 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 21:59:21.968440 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.creator() Jul 13 21:59:21.968440 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 21:59:21.968440 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application created_value = creator( Jul 13 21:59:21.969337 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 21:59:21.969337 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 21:59:21.969337 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 21:59:21.969337 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Jul 13 21:59:21.969607 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e366f84b-8daf-45b6-901c-497621c4bad8 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:21.973629 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 262/514] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Jul 13 21:59:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 21:59:21.974309 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db9f6d7-8a9a-4fd4-aad3-46deb11ac2fb None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:22.005132 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21531e9a-4bb0-497c-b20d-5ee2a1ffa86e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:22.005783 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21531e9a-4bb0-497c-b20d-5ee2a1ffa86e None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:22.006063 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21531e9a-4bb0-497c-b20d-5ee2a1ffa86e None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:22.006335 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21531e9a-4bb0-497c-b20d-5ee2a1ffa86e None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:22.070400 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21531e9a-4bb0-497c-b20d-5ee2a1ffa86e None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:22.071834 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21531e9a-4bb0-497c-b20d-5ee2a1ffa86e None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:22.072515 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db9f6d7-8a9a-4fd4-aad3-46deb11ac2fb None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:22.106750 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704c8b4f-1bcb-4ca3-97c7-a693b7fc6479 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:22.107408 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704c8b4f-1bcb-4ca3-97c7-a693b7fc6479 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:22.107680 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704c8b4f-1bcb-4ca3-97c7-a693b7fc6479 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:22.107947 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-704c8b4f-1bcb-4ca3-97c7-a693b7fc6479 None admin] PATH_INFO: `/v3/domains` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:22.108673 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-704c8b4f-1bcb-4ca3-97c7-a693b7fc6479 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 21:59:22.108979 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-704c8b4f-1bcb-4ca3-97c7-a693b7fc6479 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:22.110359 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-704c8b4f-1bcb-4ca3-97c7-a693b7fc6479 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:22.115814 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:59:22.126566 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 263/515] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Jul 13 21:59:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 21:59:22.126566 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704c8b4f-1bcb-4ca3-97c7-a693b7fc6479 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:22.142144 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:59:22.176816 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 253/516] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Jul 13 21:59:21 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 305 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 13 21:59:22.177662 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21531e9a-4bb0-497c-b20d-5ee2a1ffa86e None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:22.225037 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-704c8b4f-1bcb-4ca3-97c7-a693b7fc6479 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:22.257223 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b44996-6472-415c-a156-2d8b9cb0b89b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:22.257881 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b44996-6472-415c-a156-2d8b9cb0b89b None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:22.258129 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b44996-6472-415c-a156-2d8b9cb0b89b None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:22.258395 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b44996-6472-415c-a156-2d8b9cb0b89b None admin] PATH_INFO: `/v3/domains` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:22.259145 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b44996-6472-415c-a156-2d8b9cb0b89b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 21:59:22.259470 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b44996-6472-415c-a156-2d8b9cb0b89b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:22.260847 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b44996-6472-415c-a156-2d8b9cb0b89b None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:22.271976 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 264/517] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Jul 13 21:59:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 21:59:22.272762 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b44996-6472-415c-a156-2d8b9cb0b89b None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:22.281788 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21531e9a-4bb0-497c-b20d-5ee2a1ffa86e None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:22.317580 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d1def5-d246-4894-bd68-65218fa6eb65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:22.318223 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d1def5-d246-4894-bd68-65218fa6eb65 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:22.318523 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d1def5-d246-4894-bd68-65218fa6eb65 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:22.318786 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2d1def5-d246-4894-bd68-65218fa6eb65 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:22.333600 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:59:22.351931 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:59:22.373689 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b44996-6472-415c-a156-2d8b9cb0b89b None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:22.388687 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d1def5-d246-4894-bd68-65218fa6eb65 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:22.390133 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2d1def5-d246-4894-bd68-65218fa6eb65 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:22.407323 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f41d74-4b2f-4e17-b19a-8526d28533b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:22.408000 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f41d74-4b2f-4e17-b19a-8526d28533b6 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:22.408271 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f41d74-4b2f-4e17-b19a-8526d28533b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:22.408545 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3f41d74-4b2f-4e17-b19a-8526d28533b6 None admin] PATH_INFO: `/v3/domains` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:22.409270 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f41d74-4b2f-4e17-b19a-8526d28533b6 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 21:59:22.409575 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f41d74-4b2f-4e17-b19a-8526d28533b6 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:22.410991 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3f41d74-4b2f-4e17-b19a-8526d28533b6 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:22.422940 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 265/518] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Jul 13 21:59:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 21:59:22.423556 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f41d74-4b2f-4e17-b19a-8526d28533b6 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:22.491007 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 254/519] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Jul 13 21:59:22 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 314 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 13 21:59:22.491846 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d1def5-d246-4894-bd68-65218fa6eb65 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:22.528355 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3f41d74-4b2f-4e17-b19a-8526d28533b6 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:22.537312 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:59:22.563470 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a286e7-2711-49b2-92a3-d0802d40bfc8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:22.564280 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a286e7-2711-49b2-92a3-d0802d40bfc8 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:22.564562 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a286e7-2711-49b2-92a3-d0802d40bfc8 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:22.564836 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a286e7-2711-49b2-92a3-d0802d40bfc8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:22.571510 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a286e7-2711-49b2-92a3-d0802d40bfc8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:22.573378 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a286e7-2711-49b2-92a3-d0802d40bfc8 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:22.575540 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:59:22.589364 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application [None req-92a286e7-2711-49b2-92a3-d0802d40bfc8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 13 21:59:22.589364 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 21:59:22.589364 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 21:59:22.589364 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 21:59:22.589364 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 21:59:22.589364 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 21:59:22.589364 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 21:59:22.589364 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 21:59:22.589364 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 21:59:22.589364 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 21:59:22.589364 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 21:59:22.589364 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 21:59:22.589364 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 13 21:59:22.589364 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 13 21:59:22.589364 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 13 21:59:22.590241 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 13 21:59:22.590241 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 21:59:22.590241 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 21:59:22.590241 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 21:59:22.590241 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 21:59:22.590241 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 21:59:22.590241 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 21:59:22.590241 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 21:59:22.590241 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application with Lock( Jul 13 21:59:22.590241 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 21:59:22.590241 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._enter() Jul 13 21:59:22.590241 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 21:59:22.590241 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 21:59:22.590241 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 21:59:22.590241 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.creator() Jul 13 21:59:22.590241 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 21:59:22.590241 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application created_value = creator( Jul 13 21:59:22.591306 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 13 21:59:22.591306 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 13 21:59:22.591306 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 13 21:59:22.591306 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 13 21:59:22.591306 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 13 21:59:22.591306 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 13 21:59:22.591306 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 13 21:59:22.591306 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Jul 13 21:59:22.591306 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 266/520] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Jul 13 21:59:22 2022] GET /identity/v3/projects/service => generated 88 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 21:59:22.591772 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2d1def5-d246-4894-bd68-65218fa6eb65 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:22.591843 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a286e7-2711-49b2-92a3-d0802d40bfc8 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:22.625496 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f224f210-f0f4-4d8d-bbfe-18c93650575a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:22.626296 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f224f210-f0f4-4d8d-bbfe-18c93650575a None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:22.626598 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f224f210-f0f4-4d8d-bbfe-18c93650575a None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:22.626890 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f224f210-f0f4-4d8d-bbfe-18c93650575a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:22.636983 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f224f210-f0f4-4d8d-bbfe-18c93650575a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:22.638462 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f224f210-f0f4-4d8d-bbfe-18c93650575a None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:22.656154 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application [None req-f224f210-f0f4-4d8d-bbfe-18c93650575a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 21:59:22.656154 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 21:59:22.656154 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 21:59:22.656154 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 21:59:22.656154 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 21:59:22.656154 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 21:59:22.656154 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 21:59:22.656154 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 21:59:22.656154 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 21:59:22.656154 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 21:59:22.656154 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 21:59:22.656154 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 21:59:22.656154 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 21:59:22.656154 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 21:59:22.656154 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 21:59:22.657038 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 21:59:22.657038 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 21:59:22.657038 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 21:59:22.657038 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 21:59:22.657038 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 21:59:22.657038 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 21:59:22.657038 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 21:59:22.657038 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 21:59:22.657038 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application with Lock( Jul 13 21:59:22.657038 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 21:59:22.657038 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._enter() Jul 13 21:59:22.657038 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 21:59:22.657038 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 21:59:22.657038 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 21:59:22.657038 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.creator() Jul 13 21:59:22.657038 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 21:59:22.657038 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application created_value = creator( Jul 13 21:59:22.657910 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 21:59:22.657910 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 21:59:22.657910 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 21:59:22.657910 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Jul 13 21:59:22.657910 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 255/521] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Jul 13 21:59:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 166 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 21:59:22.658199 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f224f210-f0f4-4d8d-bbfe-18c93650575a None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:22.692155 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a286e7-2711-49b2-92a3-d0802d40bfc8 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:22.727664 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947e2163-5f74-42da-8156-39983aff946b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:22.728758 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947e2163-5f74-42da-8156-39983aff946b None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:22.729036 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947e2163-5f74-42da-8156-39983aff946b None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:22.729303 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-947e2163-5f74-42da-8156-39983aff946b None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:22.736362 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-947e2163-5f74-42da-8156-39983aff946b None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:22.737829 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-947e2163-5f74-42da-8156-39983aff946b None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:22.749302 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:59:22.752164 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application [None req-947e2163-5f74-42da-8156-39983aff946b None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 13 21:59:22.752164 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 21:59:22.752164 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 21:59:22.752164 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 21:59:22.752164 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 21:59:22.752164 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 21:59:22.752164 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 21:59:22.752164 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 21:59:22.752164 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 21:59:22.752164 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 21:59:22.752164 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 21:59:22.752164 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 21:59:22.752164 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 13 21:59:22.752164 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 13 21:59:22.752164 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 13 21:59:22.752164 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 13 21:59:22.753161 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 21:59:22.753161 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 21:59:22.753161 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 13 21:59:22.753161 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 21:59:22.753161 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 13 21:59:22.753161 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 21:59:22.753161 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 21:59:22.753161 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 21:59:22.753161 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 21:59:22.753161 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 21:59:22.753161 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 21:59:22.753161 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application with Lock( Jul 13 21:59:22.753161 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 21:59:22.753161 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._enter() Jul 13 21:59:22.753161 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 21:59:22.753161 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 21:59:22.753161 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 21:59:22.753161 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.creator() Jul 13 21:59:22.754103 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 21:59:22.754103 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application created_value = creator( Jul 13 21:59:22.754103 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 13 21:59:22.754103 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 13 21:59:22.754103 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 13 21:59:22.754103 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 13 21:59:22.754103 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 13 21:59:22.754103 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 13 21:59:22.754103 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 13 21:59:22.754103 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Jul 13 21:59:22.754103 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 267/522] 10.222.0.52 () {56 vars in 1102 bytes} [Wed Jul 13 21:59:22 2022] GET /identity/v3/users/neutron => generated 85 bytes in 162 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 21:59:22.754690 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947e2163-5f74-42da-8156-39983aff946b None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:22.760368 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f224f210-f0f4-4d8d-bbfe-18c93650575a None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:22.793800 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd17f90-eb0e-46e8-aa46-d260c732942a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:22.794504 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd17f90-eb0e-46e8-aa46-d260c732942a None admin] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:22.794782 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd17f90-eb0e-46e8-aa46-d260c732942a None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:22.795104 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd17f90-eb0e-46e8-aa46-d260c732942a None admin] PATH_INFO: `/v3/users` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:22.795844 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd17f90-eb0e-46e8-aa46-d260c732942a None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:22.797320 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd17f90-eb0e-46e8-aa46-d260c732942a None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:22.816854 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:59:22.817195 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 256/523] 10.222.0.52 () {60 vars in 1132 bytes} [Wed Jul 13 21:59:22 2022] POST /identity/v3/users => generated 247 bytes in 159 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 21:59:22.817992 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd17f90-eb0e-46e8-aa46-d260c732942a None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:22.876444 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-947e2163-5f74-42da-8156-39983aff946b None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:22.918002 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ddd5be7-9452-4392-8616-5236030f9fac None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:22.918980 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddd5be7-9452-4392-8616-5236030f9fac None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:22.919319 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddd5be7-9452-4392-8616-5236030f9fac None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:22.919596 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ddd5be7-9452-4392-8616-5236030f9fac None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:22.931666 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ddd5be7-9452-4392-8616-5236030f9fac None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:22.932448 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd17f90-eb0e-46e8-aa46-d260c732942a None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:22.933154 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ddd5be7-9452-4392-8616-5236030f9fac None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:22.954163 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application [None req-0ddd5be7-9452-4392-8616-5236030f9fac None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 21:59:22.954163 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 21:59:22.954163 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 21:59:22.954163 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 21:59:22.954163 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 21:59:22.954163 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 21:59:22.954163 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 21:59:22.954163 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 21:59:22.954163 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 21:59:22.954163 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 21:59:22.954163 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 21:59:22.954163 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 21:59:22.954163 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 21:59:22.954163 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 21:59:22.954163 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 21:59:22.955682 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 21:59:22.955682 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 21:59:22.955682 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 21:59:22.955682 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 21:59:22.955682 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 21:59:22.955682 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 21:59:22.955682 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 21:59:22.955682 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 21:59:22.955682 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application with Lock( Jul 13 21:59:22.955682 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 21:59:22.955682 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._enter() Jul 13 21:59:22.955682 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 21:59:22.955682 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 21:59:22.955682 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 21:59:22.955682 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.creator() Jul 13 21:59:22.955682 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 21:59:22.955682 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application created_value = creator( Jul 13 21:59:22.956559 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 21:59:22.956559 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 21:59:22.956559 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 21:59:22.956559 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Jul 13 21:59:22.956559 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 268/524] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Jul 13 21:59:22 2022] GET /identity/v3/domains/Default => generated 87 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 21:59:22.956559 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ddd5be7-9452-4392-8616-5236030f9fac None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:22.973035 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ffd083-95ed-4840-b5ad-695e65fc8d60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:22.974351 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ffd083-95ed-4840-b5ad-695e65fc8d60 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:22.974351 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ffd083-95ed-4840-b5ad-695e65fc8d60 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:22.974351 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6ffd083-95ed-4840-b5ad-695e65fc8d60 None admin] PATH_INFO: `/v3/projects` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:22.975073 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ffd083-95ed-4840-b5ad-695e65fc8d60 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 21:59:22.975396 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ffd083-95ed-4840-b5ad-695e65fc8d60 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:22.976853 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6ffd083-95ed-4840-b5ad-695e65fc8d60 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:22.998005 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 257/525] 10.222.0.52 () {56 vars in 1148 bytes} [Wed Jul 13 21:59:22 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 21:59:22.998463 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ffd083-95ed-4840-b5ad-695e65fc8d60 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:23.023284 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:59:23.076809 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ddd5be7-9452-4392-8616-5236030f9fac None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:23.088859 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:59:23.113640 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6ffd083-95ed-4840-b5ad-695e65fc8d60 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:23.114610 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3b4981-4f79-42c9-89d3-7c2ded0ef2dc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:23.115282 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3b4981-4f79-42c9-89d3-7c2ded0ef2dc None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:23.115556 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3b4981-4f79-42c9-89d3-7c2ded0ef2dc None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:23.115819 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad3b4981-4f79-42c9-89d3-7c2ded0ef2dc None admin] PATH_INFO: `/v3/domains` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:23.116541 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3b4981-4f79-42c9-89d3-7c2ded0ef2dc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 21:59:23.116858 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3b4981-4f79-42c9-89d3-7c2ded0ef2dc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:23.118237 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad3b4981-4f79-42c9-89d3-7c2ded0ef2dc None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:23.135132 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 269/526] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Jul 13 21:59:22 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 21:59:23.135132 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3b4981-4f79-42c9-89d3-7c2ded0ef2dc None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:23.156061 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093e8172-c042-4f59-8455-4372cdca3b9a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:23.156893 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093e8172-c042-4f59-8455-4372cdca3b9a None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:23.157178 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093e8172-c042-4f59-8455-4372cdca3b9a None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:23.157453 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-093e8172-c042-4f59-8455-4372cdca3b9a None admin] PATH_INFO: `/v3/users` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:23.158308 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093e8172-c042-4f59-8455-4372cdca3b9a None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 21:59:23.158778 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093e8172-c042-4f59-8455-4372cdca3b9a None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:23.163933 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-093e8172-c042-4f59-8455-4372cdca3b9a None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:23.183227 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 258/527] 10.222.0.52 () {56 vars in 1139 bytes} [Wed Jul 13 21:59:22 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 21:59:23.188293 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093e8172-c042-4f59-8455-4372cdca3b9a None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:23.241632 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad3b4981-4f79-42c9-89d3-7c2ded0ef2dc None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:23.258980 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:59:23.288019 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729755b5-4d75-4407-b797-92e7f6eb7ae7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:23.288691 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729755b5-4d75-4407-b797-92e7f6eb7ae7 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:23.288972 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729755b5-4d75-4407-b797-92e7f6eb7ae7 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:23.289517 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729755b5-4d75-4407-b797-92e7f6eb7ae7 None admin] PATH_INFO: `/v3/domains` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:23.290243 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729755b5-4d75-4407-b797-92e7f6eb7ae7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 21:59:23.290624 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729755b5-4d75-4407-b797-92e7f6eb7ae7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:23.292023 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729755b5-4d75-4407-b797-92e7f6eb7ae7 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:23.293066 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-093e8172-c042-4f59-8455-4372cdca3b9a None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:23.308663 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 270/528] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Jul 13 21:59:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 21:59:23.308663 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729755b5-4d75-4407-b797-92e7f6eb7ae7 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:23.337167 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93883021-9c8d-4815-a576-5727d2a77c10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:23.337859 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93883021-9c8d-4815-a576-5727d2a77c10 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:23.338266 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93883021-9c8d-4815-a576-5727d2a77c10 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:23.338557 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93883021-9c8d-4815-a576-5727d2a77c10 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:23.339462 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93883021-9c8d-4815-a576-5727d2a77c10 None admin] RBAC: Adding query filter params (role.id=70c34ea493484322911dab62b4a56e75, scope.project.id=417e3e2f1333481795a089bef2e24bea, user.id=f84281091687464990ffbca590b9343b) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 21:59:23.339829 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93883021-9c8d-4815-a576-5727d2a77c10 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:23.341276 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:59:23.341990 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93883021-9c8d-4815-a576-5727d2a77c10 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:23.358912 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 259/529] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Jul 13 21:59:23 2022] GET /identity/v3/role_assignments?user.id=f84281091687464990ffbca590b9343b&scope.project.id=417e3e2f1333481795a089bef2e24bea&role.id=70c34ea493484322911dab62b4a56e75 => generated 259 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 21:59:23.358912 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93883021-9c8d-4815-a576-5727d2a77c10 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:23.432130 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729755b5-4d75-4407-b797-92e7f6eb7ae7 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:23.470254 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948c8bc5-f515-4d2d-9a7d-64bdc263edc5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:23.471415 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948c8bc5-f515-4d2d-9a7d-64bdc263edc5 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:23.471693 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948c8bc5-f515-4d2d-9a7d-64bdc263edc5 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:23.471962 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-948c8bc5-f515-4d2d-9a7d-64bdc263edc5 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:23.482075 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-948c8bc5-f515-4d2d-9a7d-64bdc263edc5 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:23.484232 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-948c8bc5-f515-4d2d-9a7d-64bdc263edc5 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:23.489364 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93883021-9c8d-4815-a576-5727d2a77c10 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:23.504033 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application [None req-948c8bc5-f515-4d2d-9a7d-64bdc263edc5 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 13 21:59:23.504033 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 21:59:23.504033 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 21:59:23.504033 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 21:59:23.504033 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 21:59:23.504033 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 21:59:23.504033 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 21:59:23.504033 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 21:59:23.504033 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 21:59:23.504033 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 21:59:23.504033 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 21:59:23.504033 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 21:59:23.504033 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 13 21:59:23.504033 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 13 21:59:23.504033 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 13 21:59:23.504961 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 13 21:59:23.504961 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 21:59:23.504961 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 21:59:23.504961 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 21:59:23.504961 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 21:59:23.504961 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 21:59:23.504961 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 21:59:23.504961 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 21:59:23.504961 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application with Lock( Jul 13 21:59:23.504961 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 21:59:23.504961 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._enter() Jul 13 21:59:23.504961 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 21:59:23.504961 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 21:59:23.504961 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 21:59:23.504961 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.creator() Jul 13 21:59:23.504961 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 21:59:23.504961 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application created_value = creator( Jul 13 21:59:23.506272 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 13 21:59:23.506272 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 13 21:59:23.506272 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 13 21:59:23.506272 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 13 21:59:23.506272 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 13 21:59:23.506272 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 13 21:59:23.506272 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 13 21:59:23.506272 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Jul 13 21:59:23.506272 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 271/530] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Jul 13 21:59:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 21:59:23.506926 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948c8bc5-f515-4d2d-9a7d-64bdc263edc5 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:23.541187 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9313e531-da75-4ac2-afe5-5599effb34ad None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:23.542240 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9313e531-da75-4ac2-afe5-5599effb34ad None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:23.542618 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9313e531-da75-4ac2-afe5-5599effb34ad None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:23.542955 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9313e531-da75-4ac2-afe5-5599effb34ad None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:23.546333 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:59:23.622399 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9313e531-da75-4ac2-afe5-5599effb34ad None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:23.623998 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9313e531-da75-4ac2-afe5-5599effb34ad None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:23.635826 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:59:23.645798 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-948c8bc5-f515-4d2d-9a7d-64bdc263edc5 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:23.683156 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106a7ce1-0604-40ea-adc5-d8bc5b3fae6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:23.683979 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106a7ce1-0604-40ea-adc5-d8bc5b3fae6c None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:23.684266 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106a7ce1-0604-40ea-adc5-d8bc5b3fae6c None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:23.684546 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106a7ce1-0604-40ea-adc5-d8bc5b3fae6c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:23.692041 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-106a7ce1-0604-40ea-adc5-d8bc5b3fae6c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:23.693621 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-106a7ce1-0604-40ea-adc5-d8bc5b3fae6c None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:23.708899 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application [None req-106a7ce1-0604-40ea-adc5-d8bc5b3fae6c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 13 21:59:23.708899 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 21:59:23.708899 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 21:59:23.708899 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 21:59:23.708899 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 21:59:23.708899 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 21:59:23.708899 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 21:59:23.708899 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 21:59:23.708899 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 21:59:23.708899 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 21:59:23.708899 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 21:59:23.708899 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 21:59:23.708899 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 13 21:59:23.708899 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 13 21:59:23.708899 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 13 21:59:23.709792 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 13 21:59:23.709792 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 21:59:23.709792 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 21:59:23.709792 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 21:59:23.709792 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 21:59:23.709792 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 21:59:23.709792 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 21:59:23.709792 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 21:59:23.709792 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application with Lock( Jul 13 21:59:23.709792 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 21:59:23.709792 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._enter() Jul 13 21:59:23.709792 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 21:59:23.709792 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 21:59:23.709792 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 21:59:23.709792 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.creator() Jul 13 21:59:23.709792 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 21:59:23.709792 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application created_value = creator( Jul 13 21:59:23.715916 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 13 21:59:23.715916 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 13 21:59:23.715916 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 13 21:59:23.715916 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 13 21:59:23.715916 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 13 21:59:23.715916 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 13 21:59:23.715916 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 13 21:59:23.715916 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Jul 13 21:59:23.715916 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 272/531] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Jul 13 21:59:23 2022] GET /identity/v3/projects/service => generated 88 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 21:59:23.715916 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106a7ce1-0604-40ea-adc5-d8bc5b3fae6c None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:23.738467 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 260/532] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Jul 13 21:59:23 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 389 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 13 21:59:23.739771 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9313e531-da75-4ac2-afe5-5599effb34ad None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:23.774376 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:59:23.830093 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106a7ce1-0604-40ea-adc5-d8bc5b3fae6c None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:23.849466 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9313e531-da75-4ac2-afe5-5599effb34ad None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:23.869578 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f2d06d-4224-4488-9549-4a49a68e9545 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:23.870249 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f2d06d-4224-4488-9549-4a49a68e9545 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:23.870548 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f2d06d-4224-4488-9549-4a49a68e9545 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:23.870824 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0f2d06d-4224-4488-9549-4a49a68e9545 None admin] PATH_INFO: `/v3/projects` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:23.871618 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f2d06d-4224-4488-9549-4a49a68e9545 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 21:59:23.871924 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f2d06d-4224-4488-9549-4a49a68e9545 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:23.873458 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0f2d06d-4224-4488-9549-4a49a68e9545 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:23.886793 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a8f906-9749-4bbc-95ba-6d64c4868063 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:23.887787 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a8f906-9749-4bbc-95ba-6d64c4868063 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:23.888270 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a8f906-9749-4bbc-95ba-6d64c4868063 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:23.888726 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a8f906-9749-4bbc-95ba-6d64c4868063 None admin] PATH_INFO: `/v3/projects` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:23.889720 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a8f906-9749-4bbc-95ba-6d64c4868063 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 21:59:23.890213 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a8f906-9749-4bbc-95ba-6d64c4868063 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:23.891867 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a8f906-9749-4bbc-95ba-6d64c4868063 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:23.892850 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 273/533] 10.222.0.52 () {56 vars in 1148 bytes} [Wed Jul 13 21:59:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 21:59:23.893709 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f2d06d-4224-4488-9549-4a49a68e9545 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:23.907259 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:59:23.910726 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 261/534] 10.222.0.52 () {56 vars in 1148 bytes} [Wed Jul 13 21:59:23 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 21:59:23.911733 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a8f906-9749-4bbc-95ba-6d64c4868063 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:24.009429 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:59:24.017496 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a8f906-9749-4bbc-95ba-6d64c4868063 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:24.018467 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0f2d06d-4224-4488-9549-4a49a68e9545 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:24.054641 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b8c44d-49e9-4ad3-9cc5-b5b4e8776c59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:24.055549 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b8c44d-49e9-4ad3-9cc5-b5b4e8776c59 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:24.055992 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b8c44d-49e9-4ad3-9cc5-b5b4e8776c59 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:24.056490 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b8c44d-49e9-4ad3-9cc5-b5b4e8776c59 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:24.057457 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b8c44d-49e9-4ad3-9cc5-b5b4e8776c59 None admin] RBAC: Adding query filter params (role.id=70c34ea493484322911dab62b4a56e75, scope.project.id=417e3e2f1333481795a089bef2e24bea, user.id=e4d35d50e15d43d3afc59206b8165f30) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 21:59:24.057896 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b8c44d-49e9-4ad3-9cc5-b5b4e8776c59 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:24.059609 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b8c44d-49e9-4ad3-9cc5-b5b4e8776c59 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:24.059789 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94b3d1e-0c17-48b0-bdbe-5f99329fc9b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:24.060680 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94b3d1e-0c17-48b0-bdbe-5f99329fc9b2 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:24.061046 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94b3d1e-0c17-48b0-bdbe-5f99329fc9b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:24.061408 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94b3d1e-0c17-48b0-bdbe-5f99329fc9b2 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:24.066694 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 262/535] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Jul 13 21:59:23 2022] GET /identity/v3/role_assignments?user.id=e4d35d50e15d43d3afc59206b8165f30&scope.project.id=417e3e2f1333481795a089bef2e24bea&role.id=70c34ea493484322911dab62b4a56e75 => generated 259 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 21:59:24.067662 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b8c44d-49e9-4ad3-9cc5-b5b4e8776c59 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:24.073560 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94b3d1e-0c17-48b0-bdbe-5f99329fc9b2 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:24.075651 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94b3d1e-0c17-48b0-bdbe-5f99329fc9b2 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:24.095260 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application [None req-e94b3d1e-0c17-48b0-bdbe-5f99329fc9b2 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 21:59:24.095260 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 21:59:24.095260 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 21:59:24.095260 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 21:59:24.095260 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 21:59:24.095260 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 21:59:24.095260 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 21:59:24.095260 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 21:59:24.095260 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 21:59:24.095260 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 21:59:24.095260 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 21:59:24.095260 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 21:59:24.095260 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 21:59:24.095260 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 21:59:24.095260 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 21:59:24.096392 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 21:59:24.096392 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 21:59:24.096392 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 21:59:24.096392 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 21:59:24.096392 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 21:59:24.096392 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 21:59:24.096392 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 21:59:24.096392 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 21:59:24.096392 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application with Lock( Jul 13 21:59:24.096392 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 21:59:24.096392 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._enter() Jul 13 21:59:24.096392 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 21:59:24.096392 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 21:59:24.096392 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 21:59:24.096392 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.creator() Jul 13 21:59:24.096392 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 21:59:24.096392 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application created_value = creator( Jul 13 21:59:24.097426 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 21:59:24.097426 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 21:59:24.097426 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 21:59:24.097426 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Jul 13 21:59:24.103955 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 274/536] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Jul 13 21:59:23 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 21:59:24.105480 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94b3d1e-0c17-48b0-bdbe-5f99329fc9b2 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:24.185831 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b8c44d-49e9-4ad3-9cc5-b5b4e8776c59 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:24.186977 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:59:24.232247 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b39052-15cc-413b-9463-1ba418fca41b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:24.233466 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b39052-15cc-413b-9463-1ba418fca41b None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:24.234826 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b39052-15cc-413b-9463-1ba418fca41b None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:24.235462 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1b39052-15cc-413b-9463-1ba418fca41b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:24.237811 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b39052-15cc-413b-9463-1ba418fca41b None admin] RBAC: Adding query filter params (role.id=70c34ea493484322911dab62b4a56e75, scope.project.id=417e3e2f1333481795a089bef2e24bea, user.id=bc77e1ea1ada4700b809beb1c980c5e1) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 21:59:24.238589 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b39052-15cc-413b-9463-1ba418fca41b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:24.241189 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1b39052-15cc-413b-9463-1ba418fca41b None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:24.253380 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 263/537] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Jul 13 21:59:24 2022] GET /identity/v3/role_assignments?user.id=bc77e1ea1ada4700b809beb1c980c5e1&scope.project.id=417e3e2f1333481795a089bef2e24bea&role.id=70c34ea493484322911dab62b4a56e75 => generated 259 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 21:59:24.264615 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94b3d1e-0c17-48b0-bdbe-5f99329fc9b2 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:24.304306 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7989b40-0332-45a4-a273-e04422fe1f46 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:24.305416 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7989b40-0332-45a4-a273-e04422fe1f46 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:24.305882 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7989b40-0332-45a4-a273-e04422fe1f46 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:24.306395 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7989b40-0332-45a4-a273-e04422fe1f46 None admin] PATH_INFO: `/v3/domains` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:24.307409 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7989b40-0332-45a4-a273-e04422fe1f46 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 21:59:24.307904 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7989b40-0332-45a4-a273-e04422fe1f46 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:24.309519 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7989b40-0332-45a4-a273-e04422fe1f46 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:24.323643 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 275/538] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Jul 13 21:59:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 21:59:24.331159 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b39052-15cc-413b-9463-1ba418fca41b None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:24.370213 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:59:24.471573 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1b39052-15cc-413b-9463-1ba418fca41b None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:24.512547 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f12e092-dd15-4104-a7f1-e1bf5e750447 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:24.513365 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f12e092-dd15-4104-a7f1-e1bf5e750447 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:24.513648 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f12e092-dd15-4104-a7f1-e1bf5e750447 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:24.513926 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f12e092-dd15-4104-a7f1-e1bf5e750447 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:24.521523 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f12e092-dd15-4104-a7f1-e1bf5e750447 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:24.522952 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f12e092-dd15-4104-a7f1-e1bf5e750447 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:24.542370 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application [None req-2f12e092-dd15-4104-a7f1-e1bf5e750447 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 13 21:59:24.542370 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 21:59:24.542370 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 21:59:24.542370 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 21:59:24.542370 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 21:59:24.542370 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 21:59:24.542370 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 21:59:24.542370 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 21:59:24.542370 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 21:59:24.542370 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 21:59:24.542370 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 21:59:24.542370 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 21:59:24.542370 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 13 21:59:24.542370 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 13 21:59:24.542370 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 13 21:59:24.543351 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 13 21:59:24.543351 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 21:59:24.543351 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 21:59:24.543351 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 21:59:24.543351 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 21:59:24.543351 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 21:59:24.543351 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 21:59:24.543351 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 21:59:24.543351 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application with Lock( Jul 13 21:59:24.543351 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 21:59:24.543351 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._enter() Jul 13 21:59:24.543351 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 21:59:24.543351 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 21:59:24.543351 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 21:59:24.543351 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.creator() Jul 13 21:59:24.543351 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 21:59:24.543351 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application created_value = creator( Jul 13 21:59:24.544196 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 13 21:59:24.544196 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 13 21:59:24.544196 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 13 21:59:24.544196 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 13 21:59:24.544196 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 13 21:59:24.544196 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 13 21:59:24.544196 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 13 21:59:24.544196 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Jul 13 21:59:24.548098 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 264/539] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Jul 13 21:59:24 2022] GET /identity/v3/projects/service => generated 88 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 21:59:24.555110 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7989b40-0332-45a4-a273-e04422fe1f46 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:24.683367 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7989b40-0332-45a4-a273-e04422fe1f46 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:24.711802 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:59:24.730311 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f00d565d-c929-4add-9e37-95d738e8201b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:24.731027 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00d565d-c929-4add-9e37-95d738e8201b None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:24.731684 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00d565d-c929-4add-9e37-95d738e8201b None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:24.732233 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f00d565d-c929-4add-9e37-95d738e8201b None admin] PATH_INFO: `/v3/projects` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:24.733456 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00d565d-c929-4add-9e37-95d738e8201b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 21:59:24.733830 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00d565d-c929-4add-9e37-95d738e8201b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:24.735379 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f00d565d-c929-4add-9e37-95d738e8201b None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:24.752764 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 276/540] 10.222.0.52 () {56 vars in 1148 bytes} [Wed Jul 13 21:59:24 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 198 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 21:59:24.760150 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f12e092-dd15-4104-a7f1-e1bf5e750447 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:24.873564 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:59:24.898430 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f12e092-dd15-4104-a7f1-e1bf5e750447 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:24.937949 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a5c6f00-ccc6-4546-934e-c39708914435 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:24.939007 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5c6f00-ccc6-4546-934e-c39708914435 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:24.939489 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5c6f00-ccc6-4546-934e-c39708914435 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:24.939991 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a5c6f00-ccc6-4546-934e-c39708914435 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:24.941063 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5c6f00-ccc6-4546-934e-c39708914435 None admin] RBAC: Adding query filter params (role.id=70c34ea493484322911dab62b4a56e75, scope.project.id=417e3e2f1333481795a089bef2e24bea, user.id=847ec9a1bd4d4dec804318a5c845895a) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 21:59:24.941573 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5c6f00-ccc6-4546-934e-c39708914435 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:24.943230 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a5c6f00-ccc6-4546-934e-c39708914435 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:24.950213 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 265/541] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Jul 13 21:59:24 2022] GET /identity/v3/role_assignments?user.id=847ec9a1bd4d4dec804318a5c845895a&scope.project.id=417e3e2f1333481795a089bef2e24bea&role.id=70c34ea493484322911dab62b4a56e75 => generated 259 bytes in 191 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 21:59:25.058911 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a23faf-3682-4327-88ab-b39a59b94e58 None None] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:25.059443 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a23faf-3682-4327-88ab-b39a59b94e58 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:25.059443 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7a23faf-3682-4327-88ab-b39a59b94e58 None None] PATH_INFO: `/` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:25.061241 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 277/542] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jul 13 21:59:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 21:59:25.066979 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e46adef-b551-47a6-a3de-02890ea72434 None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:25.067901 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e46adef-b551-47a6-a3de-02890ea72434 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:25.068393 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e46adef-b551-47a6-a3de-02890ea72434 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:25.095805 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-2e46adef-b551-47a6-a3de-02890ea72434 None None] MFA Rules not processed for user `cc99c93da5074cf5a48bc2aa9a92c5e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:59:25.097417 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-2e46adef-b551-47a6-a3de-02890ea72434 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:59:25.189178 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 266/543] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jul 13 21:59:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 13 21:59:25.213399 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a23faf-3682-4327-88ab-b39a59b94e58 None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:25.333456 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7a23faf-3682-4327-88ab-b39a59b94e58 None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:25.376034 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4fa3ef3-c24f-4636-9463-7aa57d5d4781 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:25.376605 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fa3ef3-c24f-4636-9463-7aa57d5d4781 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:25.376884 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fa3ef3-c24f-4636-9463-7aa57d5d4781 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:25.377150 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4fa3ef3-c24f-4636-9463-7aa57d5d4781 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:25.392572 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4fa3ef3-c24f-4636-9463-7aa57d5d4781 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 21:59:25.400329 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application [None req-e4fa3ef3-c24f-4636-9463-7aa57d5d4781 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 13 21:59:25.400329 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 21:59:25.400329 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 21:59:25.400329 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 21:59:25.400329 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 21:59:25.400329 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 21:59:25.400329 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 21:59:25.400329 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 21:59:25.400329 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 21:59:25.400329 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 21:59:25.400329 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 21:59:25.400329 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 21:59:25.400329 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 21:59:25.400329 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 21:59:25.400329 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 21:59:25.400329 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 21:59:25.401223 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 21:59:25.401223 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 21:59:25.401223 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 21:59:25.401223 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 21:59:25.401223 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 21:59:25.401223 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 21:59:25.401223 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 21:59:25.401223 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 21:59:25.401223 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 21:59:25.401223 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 21:59:25.401223 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 21:59:25.401223 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application with Lock( Jul 13 21:59:25.401223 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 21:59:25.401223 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._enter() Jul 13 21:59:25.401223 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 21:59:25.401223 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 21:59:25.402043 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 21:59:25.402043 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.creator() Jul 13 21:59:25.402043 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 21:59:25.402043 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application created_value = creator( Jul 13 21:59:25.402043 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 21:59:25.402043 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 21:59:25.402043 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 21:59:25.402043 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 21:59:25.402043 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 21:59:25.402043 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 21:59:25.402043 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 13 21:59:25.402043 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Jul 13 21:59:25.402043 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 278/544] 10.222.0.52 () {56 vars in 1102 bytes} [Wed Jul 13 21:59:25 2022] GET /identity/v3/roles/service => generated 85 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 21:59:25.407972 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e46adef-b551-47a6-a3de-02890ea72434 None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:25.488804 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:59:25.513288 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e46adef-b551-47a6-a3de-02890ea72434 None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:25.548874 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b7f156-65c1-4be5-b205-f6aab7bd246b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:25.549549 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b7f156-65c1-4be5-b205-f6aab7bd246b None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:25.549823 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b7f156-65c1-4be5-b205-f6aab7bd246b None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:25.550091 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83b7f156-65c1-4be5-b205-f6aab7bd246b None admin] PATH_INFO: `/v3/roles` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:25.550902 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b7f156-65c1-4be5-b205-f6aab7bd246b None admin] RBAC: Adding query filter params (name=service) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 21:59:25.551204 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b7f156-65c1-4be5-b205-f6aab7bd246b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:25.552600 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-83b7f156-65c1-4be5-b205-f6aab7bd246b None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:25.561885 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 267/545] 10.222.0.52 () {56 vars in 1103 bytes} [Wed Jul 13 21:59:25 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 21:59:25.567526 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4fa3ef3-c24f-4636-9463-7aa57d5d4781 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:25.630871 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:59:25.686312 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4fa3ef3-c24f-4636-9463-7aa57d5d4781 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:25.727246 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c552e29-f203-4ba0-9bda-10b6de0ffcb5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:25.728119 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c552e29-f203-4ba0-9bda-10b6de0ffcb5 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:25.728396 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c552e29-f203-4ba0-9bda-10b6de0ffcb5 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:25.728811 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c552e29-f203-4ba0-9bda-10b6de0ffcb5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:25.742766 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c552e29-f203-4ba0-9bda-10b6de0ffcb5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:25.745055 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c552e29-f203-4ba0-9bda-10b6de0ffcb5 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:25.771998 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application [None req-6c552e29-f203-4ba0-9bda-10b6de0ffcb5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 21:59:25.771998 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 21:59:25.771998 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 21:59:25.771998 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 21:59:25.771998 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 21:59:25.771998 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 21:59:25.771998 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 21:59:25.771998 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 21:59:25.771998 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 21:59:25.771998 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 21:59:25.771998 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 21:59:25.771998 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 21:59:25.771998 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 21:59:25.771998 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 21:59:25.771998 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 21:59:25.772911 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 21:59:25.772911 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 21:59:25.772911 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 21:59:25.772911 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 21:59:25.772911 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 21:59:25.772911 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 21:59:25.772911 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 21:59:25.772911 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 21:59:25.772911 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application with Lock( Jul 13 21:59:25.772911 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 21:59:25.772911 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._enter() Jul 13 21:59:25.772911 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 21:59:25.772911 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 21:59:25.772911 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 21:59:25.772911 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.creator() Jul 13 21:59:25.772911 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 21:59:25.772911 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application created_value = creator( Jul 13 21:59:25.773761 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 21:59:25.773761 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 21:59:25.773761 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 21:59:25.773761 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Jul 13 21:59:25.773761 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 279/546] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Jul 13 21:59:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 21:59:25.779244 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b7f156-65c1-4be5-b205-f6aab7bd246b None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:25.825528 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426dfa85-80b2-4cd7-9f96-17b0b807d843 None None] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:25.825851 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426dfa85-80b2-4cd7-9f96-17b0b807d843 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:25.826084 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-426dfa85-80b2-4cd7-9f96-17b0b807d843 None None] PATH_INFO: `/` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:25.827508 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 280/547] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jul 13 21:59:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 21:59:25.841163 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39434b4-c2b0-484c-a860-8a6bcf0a5d51 None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:25.841447 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39434b4-c2b0-484c-a860-8a6bcf0a5d51 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:25.841727 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39434b4-c2b0-484c-a860-8a6bcf0a5d51 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:25.862813 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-f39434b4-c2b0-484c-a860-8a6bcf0a5d51 None None] MFA Rules not processed for user `cc99c93da5074cf5a48bc2aa9a92c5e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:59:25.864010 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-f39434b4-c2b0-484c-a860-8a6bcf0a5d51 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:59:25.931522 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83b7f156-65c1-4be5-b205-f6aab7bd246b None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:25.946595 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 281/548] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jul 13 21:59:25 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 13 21:59:25.963117 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39434b4-c2b0-484c-a860-8a6bcf0a5d51 None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:25.966329 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e80d73-d70a-45bc-bbca-fc22cb45cc19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:25.967093 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e80d73-d70a-45bc-bbca-fc22cb45cc19 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:25.967456 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e80d73-d70a-45bc-bbca-fc22cb45cc19 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:25.967786 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6e80d73-d70a-45bc-bbca-fc22cb45cc19 None admin] PATH_INFO: `/v3/domains` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:25.968577 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e80d73-d70a-45bc-bbca-fc22cb45cc19 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 21:59:25.968938 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e80d73-d70a-45bc-bbca-fc22cb45cc19 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:25.970389 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e6e80d73-d70a-45bc-bbca-fc22cb45cc19 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:25.977300 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:59:25.990510 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 268/549] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Jul 13 21:59:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 21:59:25.990780 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:59:25.992005 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e80d73-d70a-45bc-bbca-fc22cb45cc19 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:26.113319 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39434b4-c2b0-484c-a860-8a6bcf0a5d51 None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:26.126206 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6e80d73-d70a-45bc-bbca-fc22cb45cc19 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:26.163419 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8edd081-511a-4fb6-bdc7-78a4e00c1f18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:26.164233 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8edd081-511a-4fb6-bdc7-78a4e00c1f18 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:26.164513 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8edd081-511a-4fb6-bdc7-78a4e00c1f18 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:26.164784 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8edd081-511a-4fb6-bdc7-78a4e00c1f18 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:26.173093 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8edd081-511a-4fb6-bdc7-78a4e00c1f18 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:26.174576 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e8edd081-511a-4fb6-bdc7-78a4e00c1f18 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:26.192938 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application [None req-e8edd081-511a-4fb6-bdc7-78a4e00c1f18 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 13 21:59:26.192938 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 21:59:26.192938 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 21:59:26.192938 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 21:59:26.192938 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 21:59:26.192938 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 21:59:26.192938 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 21:59:26.192938 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 21:59:26.192938 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 21:59:26.192938 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 21:59:26.192938 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 21:59:26.192938 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 21:59:26.192938 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 13 21:59:26.192938 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 13 21:59:26.192938 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 13 21:59:26.192938 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 13 21:59:26.194718 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 21:59:26.194718 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 21:59:26.194718 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 13 21:59:26.194718 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 21:59:26.194718 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 13 21:59:26.194718 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 21:59:26.194718 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 21:59:26.194718 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 21:59:26.194718 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 21:59:26.194718 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 21:59:26.194718 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 21:59:26.194718 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application with Lock( Jul 13 21:59:26.194718 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 21:59:26.194718 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._enter() Jul 13 21:59:26.194718 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 21:59:26.194718 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 21:59:26.194718 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 21:59:26.194718 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.creator() Jul 13 21:59:26.195787 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 21:59:26.195787 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application created_value = creator( Jul 13 21:59:26.195787 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 13 21:59:26.195787 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 13 21:59:26.195787 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 13 21:59:26.195787 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 13 21:59:26.195787 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 13 21:59:26.195787 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 13 21:59:26.195787 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 13 21:59:26.195787 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Jul 13 21:59:26.195787 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 269/550] 10.222.0.52 () {56 vars in 1096 bytes} [Wed Jul 13 21:59:25 2022] GET /identity/v3/users/swift => generated 83 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 21:59:26.202243 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8edd081-511a-4fb6-bdc7-78a4e00c1f18 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:26.222752 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0bc7e9-842f-420e-829c-1b62b0cfb14f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:26.223571 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0bc7e9-842f-420e-829c-1b62b0cfb14f None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:26.223852 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0bc7e9-842f-420e-829c-1b62b0cfb14f None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:26.224125 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f0bc7e9-842f-420e-829c-1b62b0cfb14f None admin] PATH_INFO: `/v3/roles/service` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:26.243563 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f0bc7e9-842f-420e-829c-1b62b0cfb14f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 21:59:26.251385 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application [None req-2f0bc7e9-842f-420e-829c-1b62b0cfb14f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 13 21:59:26.251385 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 21:59:26.251385 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 21:59:26.251385 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 21:59:26.251385 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 21:59:26.251385 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 21:59:26.251385 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 21:59:26.251385 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 21:59:26.251385 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 21:59:26.251385 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 21:59:26.251385 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 21:59:26.251385 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 21:59:26.251385 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 21:59:26.251385 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 21:59:26.251385 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 21:59:26.251385 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 21:59:26.253454 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 21:59:26.253454 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 21:59:26.253454 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 21:59:26.253454 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 21:59:26.253454 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 21:59:26.253454 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 21:59:26.253454 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 21:59:26.253454 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 21:59:26.253454 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 21:59:26.253454 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 21:59:26.253454 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 21:59:26.253454 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application with Lock( Jul 13 21:59:26.253454 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 21:59:26.253454 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._enter() Jul 13 21:59:26.253454 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 21:59:26.253454 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 21:59:26.255407 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 21:59:26.255407 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.creator() Jul 13 21:59:26.255407 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 21:59:26.255407 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application created_value = creator( Jul 13 21:59:26.255407 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 21:59:26.255407 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 21:59:26.255407 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 21:59:26.255407 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 21:59:26.255407 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 21:59:26.255407 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 21:59:26.255407 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 13 21:59:26.255407 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Jul 13 21:59:26.255407 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 282/551] 10.222.0.52 () {56 vars in 1102 bytes} [Wed Jul 13 21:59:25 2022] GET /identity/v3/roles/service => generated 85 bytes in 291 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 21:59:26.263429 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0bc7e9-842f-420e-829c-1b62b0cfb14f None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:26.298673 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:59:26.386167 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e8edd081-511a-4fb6-bdc7-78a4e00c1f18 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:26.431260 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:59:26.450334 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cecb631-9207-4513-b763-927f46c06558 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:26.451288 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cecb631-9207-4513-b763-927f46c06558 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:26.451562 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cecb631-9207-4513-b763-927f46c06558 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:26.451833 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cecb631-9207-4513-b763-927f46c06558 None admin] PATH_INFO: `/v3/users` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:26.454807 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cecb631-9207-4513-b763-927f46c06558 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 21:59:26.455118 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cecb631-9207-4513-b763-927f46c06558 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:26.456556 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cecb631-9207-4513-b763-927f46c06558 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:26.464958 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f0bc7e9-842f-420e-829c-1b62b0cfb14f None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:26.499221 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 270/552] 10.222.0.52 () {56 vars in 1135 bytes} [Wed Jul 13 21:59:26 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 296 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 21:59:26.500743 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cecb631-9207-4513-b763-927f46c06558 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:26.541757 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f917c0-b30c-42af-84f0-1762446f8f54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:26.541757 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f917c0-b30c-42af-84f0-1762446f8f54 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:26.545519 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f917c0-b30c-42af-84f0-1762446f8f54 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:26.545822 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36f917c0-b30c-42af-84f0-1762446f8f54 None admin] PATH_INFO: `/v3/roles` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:26.546628 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f917c0-b30c-42af-84f0-1762446f8f54 None admin] RBAC: Adding query filter params (name=service) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 21:59:26.546940 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f917c0-b30c-42af-84f0-1762446f8f54 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:26.550046 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36f917c0-b30c-42af-84f0-1762446f8f54 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:26.569862 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 283/553] 10.222.0.52 () {56 vars in 1103 bytes} [Wed Jul 13 21:59:26 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 306 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 21:59:26.573647 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f917c0-b30c-42af-84f0-1762446f8f54 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:26.650603 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cecb631-9207-4513-b763-927f46c06558 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:26.661278 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:59:26.682255 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-022f2abb-d133-4dad-b570-33e0c07da3ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:26.682961 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022f2abb-d133-4dad-b570-33e0c07da3ea None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:26.683239 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022f2abb-d133-4dad-b570-33e0c07da3ea None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:26.683512 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-022f2abb-d133-4dad-b570-33e0c07da3ea None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:26.755692 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022f2abb-d133-4dad-b570-33e0c07da3ea None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:26.756135 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36f917c0-b30c-42af-84f0-1762446f8f54 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:26.757138 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-022f2abb-d133-4dad-b570-33e0c07da3ea None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:26.797842 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f17ce6a6-7be2-4ce1-a17a-401566458f9d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:26.799185 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17ce6a6-7be2-4ce1-a17a-401566458f9d None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:26.799683 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17ce6a6-7be2-4ce1-a17a-401566458f9d None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:26.800149 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f17ce6a6-7be2-4ce1-a17a-401566458f9d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:26.811688 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17ce6a6-7be2-4ce1-a17a-401566458f9d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:26.813522 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f17ce6a6-7be2-4ce1-a17a-401566458f9d None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:26.821449 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:59:26.833581 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application [None req-f17ce6a6-7be2-4ce1-a17a-401566458f9d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 21:59:26.833581 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 21:59:26.833581 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 21:59:26.833581 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 21:59:26.833581 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 21:59:26.833581 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 21:59:26.833581 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 21:59:26.833581 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 21:59:26.833581 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 21:59:26.833581 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 21:59:26.833581 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 21:59:26.833581 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 21:59:26.833581 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 21:59:26.833581 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 21:59:26.833581 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 21:59:26.834803 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 21:59:26.834803 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 21:59:26.834803 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 21:59:26.834803 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 21:59:26.834803 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 21:59:26.834803 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 21:59:26.834803 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 21:59:26.834803 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 21:59:26.834803 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application with Lock( Jul 13 21:59:26.834803 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 21:59:26.834803 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._enter() Jul 13 21:59:26.834803 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 21:59:26.834803 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 21:59:26.834803 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 21:59:26.834803 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.creator() Jul 13 21:59:26.834803 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 21:59:26.834803 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application created_value = creator( Jul 13 21:59:26.835761 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 21:59:26.835761 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 21:59:26.835761 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 21:59:26.835761 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Jul 13 21:59:26.838941 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 284/554] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Jul 13 21:59:26 2022] GET /identity/v3/domains/Default => generated 87 bytes in 265 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 21:59:26.841088 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a306f6e6-b45a-46b5-80af-f9c57470221f None None] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:26.841524 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a306f6e6-b45a-46b5-80af-f9c57470221f None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:26.841943 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a306f6e6-b45a-46b5-80af-f9c57470221f None None] PATH_INFO: `/` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:26.844310 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 285/555] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jul 13 21:59:26 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 21:59:26.845805 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3026c23d-e507-44f9-bcc1-ed99e45df3bb None None] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:26.846379 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3026c23d-e507-44f9-bcc1-ed99e45df3bb None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:26.847100 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3026c23d-e507-44f9-bcc1-ed99e45df3bb None None] PATH_INFO: `/` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:26.848679 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 286/556] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jul 13 21:59:26 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 21:59:26.849510 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3026c23d-e507-44f9-bcc1-ed99e45df3bb None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:26.864696 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 271/557] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Jul 13 21:59:26 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 13 21:59:26.866122 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206b3068-24b8-463c-94a0-f90b43ffeb45 None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:26.866406 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206b3068-24b8-463c-94a0-f90b43ffeb45 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:26.866704 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-206b3068-24b8-463c-94a0-f90b43ffeb45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:26.884555 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-206b3068-24b8-463c-94a0-f90b43ffeb45 None None] MFA Rules not processed for user `cc99c93da5074cf5a48bc2aa9a92c5e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:59:26.885878 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-206b3068-24b8-463c-94a0-f90b43ffeb45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:59:26.897495 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:59:26.962602 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3026c23d-e507-44f9-bcc1-ed99e45df3bb None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:26.962789 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 272/558] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jul 13 21:59:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 13 21:59:26.962789 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34866ec-cff6-4b54-ad89-2eea720ee6b9 None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:26.962789 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34866ec-cff6-4b54-ad89-2eea720ee6b9 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:26.962789 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a34866ec-cff6-4b54-ad89-2eea720ee6b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:26.984214 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-a34866ec-cff6-4b54-ad89-2eea720ee6b9 None None] MFA Rules not processed for user `cc99c93da5074cf5a48bc2aa9a92c5e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:59:26.985628 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-a34866ec-cff6-4b54-ad89-2eea720ee6b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:59:27.000142 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8888017c-0dfc-4e4f-a7c9-c0068c0bead1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:27.000810 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8888017c-0dfc-4e4f-a7c9-c0068c0bead1 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:27.001093 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8888017c-0dfc-4e4f-a7c9-c0068c0bead1 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:27.001356 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8888017c-0dfc-4e4f-a7c9-c0068c0bead1 None admin] PATH_INFO: `/v3/domains` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:27.002074 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8888017c-0dfc-4e4f-a7c9-c0068c0bead1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 21:59:27.002374 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8888017c-0dfc-4e4f-a7c9-c0068c0bead1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:27.003796 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8888017c-0dfc-4e4f-a7c9-c0068c0bead1 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:27.016250 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 287/559] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Jul 13 21:59:26 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 21:59:27.016250 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8888017c-0dfc-4e4f-a7c9-c0068c0bead1 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:27.065854 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 273/560] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jul 13 21:59:26 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 13 21:59:27.065854 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34866ec-cff6-4b54-ad89-2eea720ee6b9 None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:27.089129 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:59:27.133318 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8888017c-0dfc-4e4f-a7c9-c0068c0bead1 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:27.142940 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:59:27.169946 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a34866ec-cff6-4b54-ad89-2eea720ee6b9 None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:27.172550 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6065a940-1d13-451f-aae1-232d6db90388 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:27.173363 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6065a940-1d13-451f-aae1-232d6db90388 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:27.173648 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6065a940-1d13-451f-aae1-232d6db90388 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:27.173924 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6065a940-1d13-451f-aae1-232d6db90388 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:27.184735 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6065a940-1d13-451f-aae1-232d6db90388 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:27.186406 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6065a940-1d13-451f-aae1-232d6db90388 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:27.204318 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application [None req-6065a940-1d13-451f-aae1-232d6db90388 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 21:59:27.204318 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 21:59:27.204318 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 21:59:27.204318 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 21:59:27.204318 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 21:59:27.204318 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 21:59:27.204318 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 21:59:27.204318 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 21:59:27.204318 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 21:59:27.204318 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 21:59:27.204318 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 21:59:27.204318 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 21:59:27.204318 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 21:59:27.204318 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 21:59:27.204318 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 21:59:27.205380 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 21:59:27.205380 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 21:59:27.205380 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 21:59:27.205380 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 21:59:27.205380 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 21:59:27.205380 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 21:59:27.205380 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 21:59:27.205380 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 21:59:27.205380 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application with Lock( Jul 13 21:59:27.205380 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 21:59:27.205380 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._enter() Jul 13 21:59:27.205380 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 21:59:27.205380 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 21:59:27.205380 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 21:59:27.205380 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.creator() Jul 13 21:59:27.205380 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 21:59:27.205380 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application created_value = creator( Jul 13 21:59:27.206520 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 21:59:27.206520 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 21:59:27.206520 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 21:59:27.206520 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Jul 13 21:59:27.210171 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 288/561] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Jul 13 21:59:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 21:59:27.211009 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57579fe-9c68-4e4a-8ce5-cd633698c5b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:27.211144 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6065a940-1d13-451f-aae1-232d6db90388 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:27.212197 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57579fe-9c68-4e4a-8ce5-cd633698c5b9 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:27.212705 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57579fe-9c68-4e4a-8ce5-cd633698c5b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:27.213115 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e57579fe-9c68-4e4a-8ce5-cd633698c5b9 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:27.224743 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e57579fe-9c68-4e4a-8ce5-cd633698c5b9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 21:59:27.233336 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application [None req-e57579fe-9c68-4e4a-8ce5-cd633698c5b9 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 13 21:59:27.233336 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 21:59:27.233336 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 21:59:27.233336 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 21:59:27.233336 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 21:59:27.233336 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 21:59:27.233336 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 21:59:27.233336 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 21:59:27.233336 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 21:59:27.233336 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 21:59:27.233336 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 21:59:27.233336 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 21:59:27.233336 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 21:59:27.233336 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 21:59:27.233336 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 21:59:27.233336 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 21:59:27.234583 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 21:59:27.234583 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 21:59:27.234583 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 21:59:27.234583 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 21:59:27.234583 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 21:59:27.234583 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 21:59:27.234583 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 21:59:27.234583 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 21:59:27.234583 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 21:59:27.234583 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 21:59:27.234583 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 21:59:27.234583 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application with Lock( Jul 13 21:59:27.234583 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 21:59:27.234583 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._enter() Jul 13 21:59:27.234583 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 21:59:27.234583 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 21:59:27.235434 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 21:59:27.235434 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.creator() Jul 13 21:59:27.235434 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 21:59:27.235434 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application created_value = creator( Jul 13 21:59:27.235434 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 21:59:27.235434 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 21:59:27.235434 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 21:59:27.235434 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 21:59:27.235434 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 21:59:27.235434 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 21:59:27.235434 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 13 21:59:27.235434 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Jul 13 21:59:27.235434 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 274/562] 10.222.0.52 () {56 vars in 1102 bytes} [Wed Jul 13 21:59:27 2022] GET /identity/v3/roles/service => generated 85 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 21:59:27.236127 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57579fe-9c68-4e4a-8ce5-cd633698c5b9 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:27.329710 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6065a940-1d13-451f-aae1-232d6db90388 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:27.341291 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:59:27.344111 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e57579fe-9c68-4e4a-8ce5-cd633698c5b9 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:27.366708 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de04ba5-4320-4029-84b3-07ffcf268502 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:27.367513 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de04ba5-4320-4029-84b3-07ffcf268502 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:27.367798 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de04ba5-4320-4029-84b3-07ffcf268502 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:27.368064 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1de04ba5-4320-4029-84b3-07ffcf268502 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:27.376272 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de04ba5-4320-4029-84b3-07ffcf268502 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:27.377741 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1de04ba5-4320-4029-84b3-07ffcf268502 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:27.379101 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb286cbd-92ae-4285-8c75-3ece1b759a2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:27.380026 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb286cbd-92ae-4285-8c75-3ece1b759a2a None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:27.380594 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb286cbd-92ae-4285-8c75-3ece1b759a2a None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:27.380594 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb286cbd-92ae-4285-8c75-3ece1b759a2a None admin] PATH_INFO: `/v3/roles/service` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:27.383531 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:59:27.394278 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb286cbd-92ae-4285-8c75-3ece1b759a2a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 21:59:27.394419 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application [None req-1de04ba5-4320-4029-84b3-07ffcf268502 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 13 21:59:27.394419 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 21:59:27.394419 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 21:59:27.394419 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 21:59:27.394419 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 21:59:27.394419 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 21:59:27.394419 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 21:59:27.394419 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 21:59:27.394419 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 21:59:27.394419 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 21:59:27.394419 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 21:59:27.394419 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 21:59:27.394419 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 13 21:59:27.394419 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 13 21:59:27.394419 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 13 21:59:27.394419 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 13 21:59:27.395437 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 21:59:27.395437 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 21:59:27.395437 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 13 21:59:27.395437 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 21:59:27.395437 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 13 21:59:27.395437 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 21:59:27.395437 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 21:59:27.395437 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 21:59:27.395437 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 21:59:27.395437 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 21:59:27.395437 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 21:59:27.395437 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application with Lock( Jul 13 21:59:27.395437 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 21:59:27.395437 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._enter() Jul 13 21:59:27.395437 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 21:59:27.395437 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 21:59:27.395437 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 21:59:27.395437 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.creator() Jul 13 21:59:27.396407 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 21:59:27.396407 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application created_value = creator( Jul 13 21:59:27.396407 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 13 21:59:27.396407 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 13 21:59:27.396407 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 13 21:59:27.396407 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 13 21:59:27.396407 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 13 21:59:27.396407 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 13 21:59:27.396407 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 13 21:59:27.396407 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Jul 13 21:59:27.401067 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 289/563] 10.222.0.52 () {56 vars in 1093 bytes} [Wed Jul 13 21:59:27 2022] GET /identity/v3/users/nova => generated 82 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 21:59:27.402102 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de04ba5-4320-4029-84b3-07ffcf268502 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:27.402411 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application [None req-eb286cbd-92ae-4285-8c75-3ece1b759a2a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 13 21:59:27.402411 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 21:59:27.402411 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 21:59:27.402411 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 21:59:27.402411 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 21:59:27.402411 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 21:59:27.402411 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 21:59:27.402411 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 21:59:27.402411 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 21:59:27.402411 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 21:59:27.402411 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 21:59:27.402411 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 21:59:27.402411 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 21:59:27.402411 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 21:59:27.402411 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 21:59:27.402411 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 21:59:27.403450 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 21:59:27.403450 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 21:59:27.403450 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 21:59:27.403450 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 21:59:27.403450 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 21:59:27.403450 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 21:59:27.403450 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 21:59:27.403450 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 21:59:27.403450 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 21:59:27.403450 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 21:59:27.403450 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 21:59:27.403450 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application with Lock( Jul 13 21:59:27.403450 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 21:59:27.403450 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._enter() Jul 13 21:59:27.403450 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 21:59:27.403450 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 21:59:27.405339 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 21:59:27.405339 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.creator() Jul 13 21:59:27.405339 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 21:59:27.405339 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application created_value = creator( Jul 13 21:59:27.405339 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 21:59:27.405339 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 21:59:27.405339 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 21:59:27.405339 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 21:59:27.405339 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 21:59:27.405339 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 21:59:27.405339 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 13 21:59:27.405339 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Jul 13 21:59:27.405339 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 275/564] 10.222.0.52 () {56 vars in 1102 bytes} [Wed Jul 13 21:59:27 2022] GET /identity/v3/roles/service => generated 85 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 21:59:27.405339 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb286cbd-92ae-4285-8c75-3ece1b759a2a None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:27.510596 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1de04ba5-4320-4029-84b3-07ffcf268502 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:27.510788 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb286cbd-92ae-4285-8c75-3ece1b759a2a None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:27.544843 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70bf3c5-9f96-4991-9a3f-5d860aed9fa2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:27.545513 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70bf3c5-9f96-4991-9a3f-5d860aed9fa2 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:27.545789 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70bf3c5-9f96-4991-9a3f-5d860aed9fa2 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:27.546059 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d70bf3c5-9f96-4991-9a3f-5d860aed9fa2 None admin] PATH_INFO: `/v3/roles` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:27.546849 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d70bf3c5-9f96-4991-9a3f-5d860aed9fa2 None admin] RBAC: Adding query filter params (name=service) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 21:59:27.547149 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d70bf3c5-9f96-4991-9a3f-5d860aed9fa2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:27.548539 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d70bf3c5-9f96-4991-9a3f-5d860aed9fa2 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:27.548660 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc2c20c-7526-4966-be33-3d08b12e7a4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:27.549213 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc2c20c-7526-4966-be33-3d08b12e7a4b None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:27.549490 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc2c20c-7526-4966-be33-3d08b12e7a4b None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:27.549801 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffc2c20c-7526-4966-be33-3d08b12e7a4b None admin] PATH_INFO: `/v3/domains` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:27.550561 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc2c20c-7526-4966-be33-3d08b12e7a4b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 21:59:27.550875 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc2c20c-7526-4966-be33-3d08b12e7a4b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:27.552234 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffc2c20c-7526-4966-be33-3d08b12e7a4b None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:27.559062 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 276/565] 10.222.0.52 () {56 vars in 1103 bytes} [Wed Jul 13 21:59:27 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 21:59:27.559833 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70bf3c5-9f96-4991-9a3f-5d860aed9fa2 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:27.565287 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 290/566] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Jul 13 21:59:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 21:59:27.566123 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc2c20c-7526-4966-be33-3d08b12e7a4b None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:27.596612 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:59:27.626310 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:59:27.668262 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffc2c20c-7526-4966-be33-3d08b12e7a4b None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:27.668606 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d70bf3c5-9f96-4991-9a3f-5d860aed9fa2 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:27.702090 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7ab1d00-928c-4305-8d38-700040d57605 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:27.702274 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9490a7-8c18-433d-99c9-40e73607b617 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:27.702793 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ab1d00-928c-4305-8d38-700040d57605 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:27.702901 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9490a7-8c18-433d-99c9-40e73607b617 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:27.703075 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ab1d00-928c-4305-8d38-700040d57605 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:27.703192 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9490a7-8c18-433d-99c9-40e73607b617 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:27.703349 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7ab1d00-928c-4305-8d38-700040d57605 None admin] PATH_INFO: `/v3/roles` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:27.703446 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed9490a7-8c18-433d-99c9-40e73607b617 None admin] PATH_INFO: `/v3/users` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:27.704095 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ab1d00-928c-4305-8d38-700040d57605 None admin] RBAC: Adding query filter params (name=service) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 21:59:27.704194 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9490a7-8c18-433d-99c9-40e73607b617 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 21:59:27.704399 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ab1d00-928c-4305-8d38-700040d57605 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:27.704497 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9490a7-8c18-433d-99c9-40e73607b617 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:27.705772 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7ab1d00-928c-4305-8d38-700040d57605 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:27.705871 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed9490a7-8c18-433d-99c9-40e73607b617 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:27.714154 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 291/567] 10.222.0.52 () {56 vars in 1103 bytes} [Wed Jul 13 21:59:27 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 21:59:27.715421 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0723c83-9397-47ba-964d-1fac1db59473 None None] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:27.715692 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0723c83-9397-47ba-964d-1fac1db59473 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:27.715965 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0723c83-9397-47ba-964d-1fac1db59473 None None] PATH_INFO: `/` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:27.717333 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 292/568] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jul 13 21:59:27 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 21:59:27.717859 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0723c83-9397-47ba-964d-1fac1db59473 None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:27.727733 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 277/569] 10.222.0.52 () {56 vars in 1133 bytes} [Wed Jul 13 21:59:27 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 21:59:27.728814 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9490a7-8c18-433d-99c9-40e73607b617 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:27.814149 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:59:27.824736 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed9490a7-8c18-433d-99c9-40e73607b617 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:27.826050 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0723c83-9397-47ba-964d-1fac1db59473 None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:27.841060 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:59:27.858903 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e19673a-b18d-4b82-9741-78da6f8913a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:27.859699 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e19673a-b18d-4b82-9741-78da6f8913a1 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:27.859975 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e19673a-b18d-4b82-9741-78da6f8913a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:27.860244 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e19673a-b18d-4b82-9741-78da6f8913a1 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:27.861069 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cddc8af-5417-4ae9-a09c-d67c4f296f19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:27.861870 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cddc8af-5417-4ae9-a09c-d67c4f296f19 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:27.862152 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cddc8af-5417-4ae9-a09c-d67c4f296f19 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:27.862427 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cddc8af-5417-4ae9-a09c-d67c4f296f19 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:27.866992 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e19673a-b18d-4b82-9741-78da6f8913a1 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:27.868495 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e19673a-b18d-4b82-9741-78da6f8913a1 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:27.872990 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cddc8af-5417-4ae9-a09c-d67c4f296f19 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:27.874457 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cddc8af-5417-4ae9-a09c-d67c4f296f19 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:27.883271 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application [None req-0e19673a-b18d-4b82-9741-78da6f8913a1 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 13 21:59:27.883271 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 21:59:27.883271 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 21:59:27.883271 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 21:59:27.883271 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 21:59:27.883271 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 21:59:27.883271 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 21:59:27.883271 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 21:59:27.883271 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 21:59:27.883271 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 21:59:27.883271 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 21:59:27.883271 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 21:59:27.883271 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 13 21:59:27.883271 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 13 21:59:27.883271 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 13 21:59:27.884295 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 13 21:59:27.884295 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 21:59:27.884295 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 21:59:27.884295 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 21:59:27.884295 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 21:59:27.884295 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 21:59:27.884295 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 21:59:27.884295 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 21:59:27.884295 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application with Lock( Jul 13 21:59:27.884295 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 21:59:27.884295 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._enter() Jul 13 21:59:27.884295 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 21:59:27.884295 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 21:59:27.884295 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 21:59:27.884295 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.creator() Jul 13 21:59:27.884295 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 21:59:27.884295 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application created_value = creator( Jul 13 21:59:27.885425 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 13 21:59:27.885425 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 13 21:59:27.885425 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 13 21:59:27.885425 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 13 21:59:27.885425 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 13 21:59:27.885425 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 13 21:59:27.885425 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 13 21:59:27.885425 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Jul 13 21:59:27.885425 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 278/570] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Jul 13 21:59:27 2022] GET /identity/v3/projects/service => generated 88 bytes in 156 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 21:59:27.885425 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e19673a-b18d-4b82-9741-78da6f8913a1 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:27.892824 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application [None req-8cddc8af-5417-4ae9-a09c-d67c4f296f19 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 21:59:27.892824 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 21:59:27.892824 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 21:59:27.892824 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 21:59:27.892824 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 21:59:27.892824 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 21:59:27.892824 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 21:59:27.892824 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 21:59:27.892824 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 21:59:27.892824 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 21:59:27.892824 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 21:59:27.892824 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 21:59:27.892824 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 21:59:27.892824 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 21:59:27.892824 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 21:59:27.893724 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 21:59:27.893724 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 21:59:27.893724 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 21:59:27.893724 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 21:59:27.893724 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 21:59:27.893724 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 21:59:27.893724 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 21:59:27.893724 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 21:59:27.893724 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application with Lock( Jul 13 21:59:27.893724 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 21:59:27.893724 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._enter() Jul 13 21:59:27.893724 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 21:59:27.893724 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 21:59:27.893724 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 21:59:27.893724 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.creator() Jul 13 21:59:27.893724 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 21:59:27.893724 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application created_value = creator( Jul 13 21:59:27.894887 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 21:59:27.894887 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 21:59:27.894887 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 21:59:27.894887 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Jul 13 21:59:27.894887 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 293/571] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Jul 13 21:59:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 21:59:27.895718 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b554023-d2d1-4caa-856e-c67d473583e7 None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:27.895718 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b554023-d2d1-4caa-856e-c67d473583e7 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:27.895999 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b554023-d2d1-4caa-856e-c67d473583e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:27.913101 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-5b554023-d2d1-4caa-856e-c67d473583e7 None None] MFA Rules not processed for user `cc99c93da5074cf5a48bc2aa9a92c5e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:59:27.914285 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-5b554023-d2d1-4caa-856e-c67d473583e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:59:27.980889 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 294/572] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jul 13 21:59:27 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 13 21:59:27.981731 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b554023-d2d1-4caa-856e-c67d473583e7 None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:27.998872 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e19673a-b18d-4b82-9741-78da6f8913a1 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:28.034159 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc64b81-bcc3-40f0-97fb-9347e1741138 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:28.034986 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc64b81-bcc3-40f0-97fb-9347e1741138 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:28.035263 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc64b81-bcc3-40f0-97fb-9347e1741138 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:28.035537 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dc64b81-bcc3-40f0-97fb-9347e1741138 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:28.046225 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc64b81-bcc3-40f0-97fb-9347e1741138 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:28.047740 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dc64b81-bcc3-40f0-97fb-9347e1741138 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:28.054562 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:59:28.065189 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application [None req-4dc64b81-bcc3-40f0-97fb-9347e1741138 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 21:59:28.065189 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 21:59:28.065189 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 21:59:28.065189 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 21:59:28.065189 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 21:59:28.065189 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 21:59:28.065189 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 21:59:28.065189 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 21:59:28.065189 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 21:59:28.065189 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 21:59:28.065189 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 21:59:28.065189 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 21:59:28.065189 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 21:59:28.065189 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 21:59:28.065189 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 21:59:28.066128 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 21:59:28.066128 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 21:59:28.066128 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 21:59:28.066128 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 21:59:28.066128 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 21:59:28.066128 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 21:59:28.066128 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 21:59:28.066128 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 21:59:28.066128 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application with Lock( Jul 13 21:59:28.066128 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 21:59:28.066128 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._enter() Jul 13 21:59:28.066128 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 21:59:28.066128 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 21:59:28.066128 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 21:59:28.066128 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.creator() Jul 13 21:59:28.066128 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 21:59:28.066128 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application created_value = creator( Jul 13 21:59:28.067046 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 21:59:28.067046 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 21:59:28.067046 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 21:59:28.067046 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Jul 13 21:59:28.067046 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 279/573] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Jul 13 21:59:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 21:59:28.067584 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc64b81-bcc3-40f0-97fb-9347e1741138 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:28.080746 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b554023-d2d1-4caa-856e-c67d473583e7 None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:28.099812 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:59:28.116374 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ddf47c-f138-40d1-a992-42229f8b83b6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:28.117179 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ddf47c-f138-40d1-a992-42229f8b83b6 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:28.117454 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ddf47c-f138-40d1-a992-42229f8b83b6 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:28.117776 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76ddf47c-f138-40d1-a992-42229f8b83b6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:28.128465 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ddf47c-f138-40d1-a992-42229f8b83b6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:28.129934 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76ddf47c-f138-40d1-a992-42229f8b83b6 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:28.147991 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application [None req-76ddf47c-f138-40d1-a992-42229f8b83b6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 21:59:28.147991 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 21:59:28.147991 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 21:59:28.147991 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 21:59:28.147991 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 21:59:28.147991 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 21:59:28.147991 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 21:59:28.147991 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 21:59:28.147991 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 21:59:28.147991 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 21:59:28.147991 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 21:59:28.147991 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 21:59:28.147991 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 21:59:28.147991 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 21:59:28.147991 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 21:59:28.149300 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 21:59:28.149300 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 21:59:28.149300 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 21:59:28.149300 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 21:59:28.149300 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 21:59:28.149300 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 21:59:28.149300 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 21:59:28.149300 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 21:59:28.149300 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application with Lock( Jul 13 21:59:28.149300 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 21:59:28.149300 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._enter() Jul 13 21:59:28.149300 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 21:59:28.149300 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 21:59:28.149300 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 21:59:28.149300 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.creator() Jul 13 21:59:28.149300 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 21:59:28.149300 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application created_value = creator( Jul 13 21:59:28.150468 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 21:59:28.150468 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 21:59:28.150468 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 21:59:28.150468 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Jul 13 21:59:28.150468 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 295/574] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Jul 13 21:59:27 2022] GET /identity/v3/domains/Default => generated 87 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 21:59:28.150782 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ddf47c-f138-40d1-a992-42229f8b83b6 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:28.176816 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dc64b81-bcc3-40f0-97fb-9347e1741138 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:28.214000 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7ff145-9997-49c3-af3c-7a6ddb9b99a2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:28.214685 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7ff145-9997-49c3-af3c-7a6ddb9b99a2 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:28.214963 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7ff145-9997-49c3-af3c-7a6ddb9b99a2 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:28.215231 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f7ff145-9997-49c3-af3c-7a6ddb9b99a2 None admin] PATH_INFO: `/v3/projects` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:28.216031 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7ff145-9997-49c3-af3c-7a6ddb9b99a2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 21:59:28.216336 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7ff145-9997-49c3-af3c-7a6ddb9b99a2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:28.217776 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f7ff145-9997-49c3-af3c-7a6ddb9b99a2 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:28.229733 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 280/575] 10.222.0.52 () {56 vars in 1148 bytes} [Wed Jul 13 21:59:28 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 21:59:28.230658 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7ff145-9997-49c3-af3c-7a6ddb9b99a2 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:28.251776 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76ddf47c-f138-40d1-a992-42229f8b83b6 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:28.277872 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:59:28.287123 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44691fbf-8d48-4bc6-b8d9-d64dd4ec5d98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:28.287797 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44691fbf-8d48-4bc6-b8d9-d64dd4ec5d98 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:28.288077 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44691fbf-8d48-4bc6-b8d9-d64dd4ec5d98 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:28.288354 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44691fbf-8d48-4bc6-b8d9-d64dd4ec5d98 None admin] PATH_INFO: `/v3/domains` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:28.289090 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44691fbf-8d48-4bc6-b8d9-d64dd4ec5d98 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 21:59:28.289396 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44691fbf-8d48-4bc6-b8d9-d64dd4ec5d98 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:28.291078 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44691fbf-8d48-4bc6-b8d9-d64dd4ec5d98 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:28.303760 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 296/576] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Jul 13 21:59:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 21:59:28.304555 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44691fbf-8d48-4bc6-b8d9-d64dd4ec5d98 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:28.321622 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:59:28.335040 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f7ff145-9997-49c3-af3c-7a6ddb9b99a2 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:28.373880 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318327f8-9da5-47c6-ab79-864ee6ccf4a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:28.374717 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318327f8-9da5-47c6-ab79-864ee6ccf4a8 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:28.374995 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318327f8-9da5-47c6-ab79-864ee6ccf4a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:28.375267 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-318327f8-9da5-47c6-ab79-864ee6ccf4a8 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:28.386015 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-318327f8-9da5-47c6-ab79-864ee6ccf4a8 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 21:59:28.393849 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application [None req-318327f8-9da5-47c6-ab79-864ee6ccf4a8 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 13 21:59:28.393849 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 21:59:28.393849 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 21:59:28.393849 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 21:59:28.393849 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 21:59:28.393849 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 21:59:28.393849 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 21:59:28.393849 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 21:59:28.393849 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 21:59:28.393849 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 21:59:28.393849 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 21:59:28.393849 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 21:59:28.393849 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 21:59:28.393849 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 21:59:28.393849 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 21:59:28.393849 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 21:59:28.394883 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 21:59:28.394883 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 21:59:28.394883 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 21:59:28.394883 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 21:59:28.394883 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 21:59:28.394883 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 21:59:28.394883 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 21:59:28.394883 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 21:59:28.394883 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 21:59:28.394883 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 21:59:28.394883 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 21:59:28.394883 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application with Lock( Jul 13 21:59:28.394883 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 21:59:28.394883 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._enter() Jul 13 21:59:28.394883 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 21:59:28.394883 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 21:59:28.395901 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 21:59:28.395901 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.creator() Jul 13 21:59:28.395901 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 21:59:28.395901 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application created_value = creator( Jul 13 21:59:28.395901 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 21:59:28.395901 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 21:59:28.395901 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 21:59:28.395901 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 21:59:28.395901 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 21:59:28.395901 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 21:59:28.395901 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 13 21:59:28.395901 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Jul 13 21:59:28.395901 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 281/577] 10.222.0.52 () {56 vars in 1102 bytes} [Wed Jul 13 21:59:28 2022] GET /identity/v3/roles/service => generated 85 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 21:59:28.395901 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318327f8-9da5-47c6-ab79-864ee6ccf4a8 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:28.406842 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44691fbf-8d48-4bc6-b8d9-d64dd4ec5d98 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:28.442856 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab633966-59a4-4250-a683-f96ddb0e7f30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:28.443649 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab633966-59a4-4250-a683-f96ddb0e7f30 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:28.443926 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab633966-59a4-4250-a683-f96ddb0e7f30 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:28.444196 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab633966-59a4-4250-a683-f96ddb0e7f30 None admin] PATH_INFO: `/v3/domains` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:28.444920 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab633966-59a4-4250-a683-f96ddb0e7f30 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 21:59:28.445231 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab633966-59a4-4250-a683-f96ddb0e7f30 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:28.446836 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab633966-59a4-4250-a683-f96ddb0e7f30 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:28.458333 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 297/578] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Jul 13 21:59:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 21:59:28.459145 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab633966-59a4-4250-a683-f96ddb0e7f30 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:28.494841 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-318327f8-9da5-47c6-ab79-864ee6ccf4a8 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:28.504433 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:59:28.527154 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0bbf8d5-74f8-4132-b67a-c58af94766e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:28.527822 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bbf8d5-74f8-4132-b67a-c58af94766e2 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:28.528094 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bbf8d5-74f8-4132-b67a-c58af94766e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:28.528362 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0bbf8d5-74f8-4132-b67a-c58af94766e2 None admin] PATH_INFO: `/v3/domains` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:28.529078 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0bbf8d5-74f8-4132-b67a-c58af94766e2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 21:59:28.529375 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0bbf8d5-74f8-4132-b67a-c58af94766e2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:28.530779 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0bbf8d5-74f8-4132-b67a-c58af94766e2 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:28.546033 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 282/579] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Jul 13 21:59:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 21:59:28.546884 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0bbf8d5-74f8-4132-b67a-c58af94766e2 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:28.557578 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:59:28.564213 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab633966-59a4-4250-a683-f96ddb0e7f30 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:28.598581 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e16a30d-01ea-4c51-96f9-87fcca95a033 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:28.599210 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e16a30d-01ea-4c51-96f9-87fcca95a033 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:28.599488 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e16a30d-01ea-4c51-96f9-87fcca95a033 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:28.599762 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e16a30d-01ea-4c51-96f9-87fcca95a033 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:28.600595 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e16a30d-01ea-4c51-96f9-87fcca95a033 None admin] RBAC: Adding query filter params (role.id=70c34ea493484322911dab62b4a56e75, scope.project.id=417e3e2f1333481795a089bef2e24bea, user.id=1eb1525233ba48c99b15d4cb75db4822) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 21:59:28.600912 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e16a30d-01ea-4c51-96f9-87fcca95a033 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:28.602369 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e16a30d-01ea-4c51-96f9-87fcca95a033 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:28.607776 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 298/580] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Jul 13 21:59:28 2022] GET /identity/v3/role_assignments?user.id=1eb1525233ba48c99b15d4cb75db4822&scope.project.id=417e3e2f1333481795a089bef2e24bea&role.id=70c34ea493484322911dab62b4a56e75 => generated 259 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 21:59:28.608542 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e16a30d-01ea-4c51-96f9-87fcca95a033 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:28.651030 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0bbf8d5-74f8-4132-b67a-c58af94766e2 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:28.688203 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d17568-3d8a-4112-96d5-bb89071ee839 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:28.689077 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d17568-3d8a-4112-96d5-bb89071ee839 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:28.689374 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d17568-3d8a-4112-96d5-bb89071ee839 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:28.689644 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3d17568-3d8a-4112-96d5-bb89071ee839 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:28.698074 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d17568-3d8a-4112-96d5-bb89071ee839 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:28.699988 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3d17568-3d8a-4112-96d5-bb89071ee839 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:28.716084 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application [None req-a3d17568-3d8a-4112-96d5-bb89071ee839 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 13 21:59:28.716084 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 21:59:28.716084 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 21:59:28.716084 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 21:59:28.716084 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 21:59:28.716084 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 21:59:28.716084 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 21:59:28.716084 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 21:59:28.716084 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 21:59:28.716084 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 21:59:28.716084 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 21:59:28.716084 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 21:59:28.716084 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 13 21:59:28.716084 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 13 21:59:28.716084 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 13 21:59:28.716084 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 13 21:59:28.717097 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 21:59:28.717097 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 21:59:28.717097 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 13 21:59:28.717097 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 21:59:28.717097 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 13 21:59:28.717097 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 21:59:28.717097 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 21:59:28.717097 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 21:59:28.717097 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 21:59:28.717097 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 21:59:28.717097 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 21:59:28.717097 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application with Lock( Jul 13 21:59:28.717097 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 21:59:28.717097 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._enter() Jul 13 21:59:28.717097 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 21:59:28.717097 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 21:59:28.717097 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 21:59:28.717097 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.creator() Jul 13 21:59:28.723445 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 21:59:28.723445 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application created_value = creator( Jul 13 21:59:28.723445 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 13 21:59:28.723445 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 13 21:59:28.723445 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 13 21:59:28.723445 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 13 21:59:28.723445 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 13 21:59:28.723445 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 13 21:59:28.723445 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 13 21:59:28.723445 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Jul 13 21:59:28.724086 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e16a30d-01ea-4c51-96f9-87fcca95a033 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:28.724164 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 283/581] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Jul 13 21:59:28 2022] GET /identity/v3/users/glance => generated 84 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 21:59:28.724164 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d17568-3d8a-4112-96d5-bb89071ee839 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:28.730107 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:59:28.757699 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630d34e2-5391-450a-9dc1-a4b353f494a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:28.758714 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630d34e2-5391-450a-9dc1-a4b353f494a1 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:28.759003 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630d34e2-5391-450a-9dc1-a4b353f494a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:28.759354 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-630d34e2-5391-450a-9dc1-a4b353f494a1 None admin] PATH_INFO: `/v3/roles` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:28.760114 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-630d34e2-5391-450a-9dc1-a4b353f494a1 None admin] RBAC: Adding query filter params (name=service) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 21:59:28.760416 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-630d34e2-5391-450a-9dc1-a4b353f494a1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:28.761805 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-630d34e2-5391-450a-9dc1-a4b353f494a1 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:28.770522 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 299/582] 10.222.0.52 () {56 vars in 1103 bytes} [Wed Jul 13 21:59:28 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 21:59:28.771332 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630d34e2-5391-450a-9dc1-a4b353f494a1 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:28.798439 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:59:28.836298 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3d17568-3d8a-4112-96d5-bb89071ee839 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:28.872138 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6391bd16-d120-4d49-818d-e73bc0294e33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:28.872860 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6391bd16-d120-4d49-818d-e73bc0294e33 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:28.873138 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6391bd16-d120-4d49-818d-e73bc0294e33 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:28.873408 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6391bd16-d120-4d49-818d-e73bc0294e33 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:28.880691 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6391bd16-d120-4d49-818d-e73bc0294e33 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:28.882118 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6391bd16-d120-4d49-818d-e73bc0294e33 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:28.882998 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-630d34e2-5391-450a-9dc1-a4b353f494a1 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:28.896762 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application [None req-6391bd16-d120-4d49-818d-e73bc0294e33 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 13 21:59:28.896762 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 21:59:28.896762 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 21:59:28.896762 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 21:59:28.896762 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 21:59:28.896762 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 21:59:28.896762 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 21:59:28.896762 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 21:59:28.896762 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 21:59:28.896762 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 21:59:28.896762 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 21:59:28.896762 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 21:59:28.896762 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 13 21:59:28.896762 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 13 21:59:28.896762 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 13 21:59:28.896762 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 13 21:59:28.897762 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 21:59:28.897762 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 21:59:28.897762 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 13 21:59:28.897762 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 21:59:28.897762 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 13 21:59:28.897762 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 21:59:28.897762 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 21:59:28.897762 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 21:59:28.897762 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 21:59:28.897762 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 21:59:28.897762 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 21:59:28.897762 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application with Lock( Jul 13 21:59:28.897762 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 21:59:28.897762 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._enter() Jul 13 21:59:28.897762 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 21:59:28.897762 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 21:59:28.897762 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 21:59:28.897762 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.creator() Jul 13 21:59:28.898758 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 21:59:28.898758 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application created_value = creator( Jul 13 21:59:28.898758 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 13 21:59:28.898758 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 13 21:59:28.898758 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 13 21:59:28.898758 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 13 21:59:28.898758 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 13 21:59:28.898758 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 13 21:59:28.898758 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 13 21:59:28.898758 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Jul 13 21:59:28.898758 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 284/583] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Jul 13 21:59:28 2022] GET /identity/v3/users/cinder => generated 84 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 21:59:28.898758 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6391bd16-d120-4d49-818d-e73bc0294e33 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:28.923214 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84252ff-2ee3-4774-9ea1-e930871a63b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:28.924025 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84252ff-2ee3-4774-9ea1-e930871a63b7 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:28.924300 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84252ff-2ee3-4774-9ea1-e930871a63b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:28.924568 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e84252ff-2ee3-4774-9ea1-e930871a63b7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:28.931759 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e84252ff-2ee3-4774-9ea1-e930871a63b7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:28.933209 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e84252ff-2ee3-4774-9ea1-e930871a63b7 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:28.947485 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application [None req-e84252ff-2ee3-4774-9ea1-e930871a63b7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 13 21:59:28.947485 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 21:59:28.947485 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 21:59:28.947485 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 21:59:28.947485 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 21:59:28.947485 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 21:59:28.947485 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 21:59:28.947485 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 21:59:28.947485 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 21:59:28.947485 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 21:59:28.947485 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 21:59:28.947485 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 21:59:28.947485 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 13 21:59:28.947485 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 13 21:59:28.947485 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 13 21:59:28.948565 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 13 21:59:28.948565 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 21:59:28.948565 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 21:59:28.948565 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 21:59:28.948565 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 21:59:28.948565 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 21:59:28.948565 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 21:59:28.948565 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 21:59:28.948565 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application with Lock( Jul 13 21:59:28.948565 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 21:59:28.948565 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._enter() Jul 13 21:59:28.948565 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 21:59:28.948565 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 21:59:28.948565 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 21:59:28.948565 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.creator() Jul 13 21:59:28.948565 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 21:59:28.948565 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application created_value = creator( Jul 13 21:59:28.950541 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 13 21:59:28.950541 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 13 21:59:28.950541 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 13 21:59:28.950541 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 13 21:59:28.950541 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 13 21:59:28.950541 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 13 21:59:28.950541 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 13 21:59:28.950541 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Jul 13 21:59:28.950541 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 300/584] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Jul 13 21:59:28 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 21:59:28.950541 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84252ff-2ee3-4774-9ea1-e930871a63b7 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:28.982095 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:59:29.014660 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6391bd16-d120-4d49-818d-e73bc0294e33 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:29.042913 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:59:29.052394 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383783cd-14c3-415d-bf78-506ab32d55a0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:29.053071 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383783cd-14c3-415d-bf78-506ab32d55a0 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:29.053352 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383783cd-14c3-415d-bf78-506ab32d55a0 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:29.053625 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-383783cd-14c3-415d-bf78-506ab32d55a0 None admin] PATH_INFO: `/v3/users` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:29.054539 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383783cd-14c3-415d-bf78-506ab32d55a0 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 21:59:29.054850 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383783cd-14c3-415d-bf78-506ab32d55a0 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:29.056514 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-383783cd-14c3-415d-bf78-506ab32d55a0 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:29.058704 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e84252ff-2ee3-4774-9ea1-e930871a63b7 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:29.078512 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 285/585] 10.222.0.52 () {56 vars in 1137 bytes} [Wed Jul 13 21:59:28 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 21:59:29.078512 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383783cd-14c3-415d-bf78-506ab32d55a0 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:29.101181 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3618daf4-e64e-46d6-bde0-e6868acc40e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:29.102083 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3618daf4-e64e-46d6-bde0-e6868acc40e4 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:29.102375 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3618daf4-e64e-46d6-bde0-e6868acc40e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:29.102679 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3618daf4-e64e-46d6-bde0-e6868acc40e4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:29.114916 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3618daf4-e64e-46d6-bde0-e6868acc40e4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:29.116651 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3618daf4-e64e-46d6-bde0-e6868acc40e4 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:29.136270 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application [None req-3618daf4-e64e-46d6-bde0-e6868acc40e4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 21:59:29.136270 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 21:59:29.136270 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 21:59:29.136270 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 21:59:29.136270 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 21:59:29.136270 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 21:59:29.136270 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 21:59:29.136270 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 21:59:29.136270 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 21:59:29.136270 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 21:59:29.136270 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 21:59:29.136270 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 21:59:29.136270 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 21:59:29.136270 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 21:59:29.136270 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 21:59:29.137368 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 21:59:29.137368 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 21:59:29.137368 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 21:59:29.137368 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 21:59:29.137368 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 21:59:29.137368 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 21:59:29.137368 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 21:59:29.137368 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 21:59:29.137368 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application with Lock( Jul 13 21:59:29.137368 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 21:59:29.137368 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._enter() Jul 13 21:59:29.137368 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 21:59:29.137368 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 21:59:29.137368 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 21:59:29.137368 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.creator() Jul 13 21:59:29.137368 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 21:59:29.137368 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application created_value = creator( Jul 13 21:59:29.138518 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 21:59:29.138518 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 21:59:29.138518 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 21:59:29.138518 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Jul 13 21:59:29.143805 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 301/586] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Jul 13 21:59:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 21:59:29.143805 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3618daf4-e64e-46d6-bde0-e6868acc40e4 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:29.193367 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-383783cd-14c3-415d-bf78-506ab32d55a0 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:29.227757 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:59:29.234439 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904ed07c-b447-4a28-8401-fc097fc31cd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:29.235174 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904ed07c-b447-4a28-8401-fc097fc31cd8 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:29.235455 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904ed07c-b447-4a28-8401-fc097fc31cd8 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:29.235726 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-904ed07c-b447-4a28-8401-fc097fc31cd8 None admin] PATH_INFO: `/v3/users` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:29.236581 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904ed07c-b447-4a28-8401-fc097fc31cd8 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 21:59:29.236890 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904ed07c-b447-4a28-8401-fc097fc31cd8 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:29.238335 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-904ed07c-b447-4a28-8401-fc097fc31cd8 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:29.246981 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3618daf4-e64e-46d6-bde0-e6868acc40e4 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:29.262292 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 286/587] 10.222.0.52 () {56 vars in 1137 bytes} [Wed Jul 13 21:59:29 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 21:59:29.266843 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904ed07c-b447-4a28-8401-fc097fc31cd8 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:29.290439 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3163d68b-cd0f-4dee-bb0b-0c699fb9fc51 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:29.291484 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3163d68b-cd0f-4dee-bb0b-0c699fb9fc51 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:29.291944 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3163d68b-cd0f-4dee-bb0b-0c699fb9fc51 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:29.292186 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3163d68b-cd0f-4dee-bb0b-0c699fb9fc51 None admin] PATH_INFO: `/v3/projects` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:29.292990 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3163d68b-cd0f-4dee-bb0b-0c699fb9fc51 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 21:59:29.293395 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3163d68b-cd0f-4dee-bb0b-0c699fb9fc51 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:29.294766 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3163d68b-cd0f-4dee-bb0b-0c699fb9fc51 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:29.309245 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 302/588] 10.222.0.52 () {56 vars in 1148 bytes} [Wed Jul 13 21:59:29 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 21:59:29.310426 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3163d68b-cd0f-4dee-bb0b-0c699fb9fc51 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:29.321920 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:59:29.378656 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-904ed07c-b447-4a28-8401-fc097fc31cd8 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:29.414325 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4449d188-5b0a-40dd-a3f1-b3244fa857d8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:29.415167 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4449d188-5b0a-40dd-a3f1-b3244fa857d8 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:29.415497 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4449d188-5b0a-40dd-a3f1-b3244fa857d8 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:29.415772 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4449d188-5b0a-40dd-a3f1-b3244fa857d8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:29.421218 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3163d68b-cd0f-4dee-bb0b-0c699fb9fc51 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:29.426754 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4449d188-5b0a-40dd-a3f1-b3244fa857d8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:29.428206 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4449d188-5b0a-40dd-a3f1-b3244fa857d8 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:29.446049 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application [None req-4449d188-5b0a-40dd-a3f1-b3244fa857d8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 21:59:29.446049 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 21:59:29.446049 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 21:59:29.446049 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 21:59:29.446049 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 21:59:29.446049 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 21:59:29.446049 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 21:59:29.446049 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 21:59:29.446049 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 21:59:29.446049 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 21:59:29.446049 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 21:59:29.446049 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 21:59:29.446049 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 21:59:29.446049 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 21:59:29.446049 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 21:59:29.447212 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 21:59:29.447212 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 21:59:29.447212 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 21:59:29.447212 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 21:59:29.447212 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 21:59:29.447212 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 21:59:29.447212 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 21:59:29.447212 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 21:59:29.447212 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application with Lock( Jul 13 21:59:29.447212 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 21:59:29.447212 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._enter() Jul 13 21:59:29.447212 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 21:59:29.447212 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 21:59:29.447212 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 21:59:29.447212 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.creator() Jul 13 21:59:29.447212 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 21:59:29.447212 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application created_value = creator( Jul 13 21:59:29.448365 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 21:59:29.448365 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 21:59:29.448365 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 21:59:29.448365 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Jul 13 21:59:29.452232 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 287/589] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Jul 13 21:59:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 21:59:29.452949 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4449d188-5b0a-40dd-a3f1-b3244fa857d8 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:29.461724 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a73b3e1-f182-4a4e-8ef9-aacc9cb97933 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:29.462661 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a73b3e1-f182-4a4e-8ef9-aacc9cb97933 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:29.463206 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a73b3e1-f182-4a4e-8ef9-aacc9cb97933 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:29.463488 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a73b3e1-f182-4a4e-8ef9-aacc9cb97933 None admin] PATH_INFO: `/v3/domains` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:29.464227 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a73b3e1-f182-4a4e-8ef9-aacc9cb97933 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 21:59:29.464656 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a73b3e1-f182-4a4e-8ef9-aacc9cb97933 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:29.466125 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a73b3e1-f182-4a4e-8ef9-aacc9cb97933 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:29.478423 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 303/590] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Jul 13 21:59:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 21:59:29.479530 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a73b3e1-f182-4a4e-8ef9-aacc9cb97933 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:29.484110 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:59:29.560947 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:59:29.566812 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4449d188-5b0a-40dd-a3f1-b3244fa857d8 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:29.600495 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a73b3e1-f182-4a4e-8ef9-aacc9cb97933 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:29.606896 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5effac-43cc-45a3-a092-280e4f3b906e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:29.607940 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5effac-43cc-45a3-a092-280e4f3b906e None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:29.608313 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5effac-43cc-45a3-a092-280e4f3b906e None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:29.608678 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd5effac-43cc-45a3-a092-280e4f3b906e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:29.619358 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd5effac-43cc-45a3-a092-280e4f3b906e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:29.621032 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd5effac-43cc-45a3-a092-280e4f3b906e None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:29.636898 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4575a604-3529-4cc5-ace6-e962f8c52668 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:29.637797 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4575a604-3529-4cc5-ace6-e962f8c52668 None admin] REQUEST_METHOD: `PUT` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:29.638076 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4575a604-3529-4cc5-ace6-e962f8c52668 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:29.638347 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4575a604-3529-4cc5-ace6-e962f8c52668 None admin] PATH_INFO: `/v3/projects/417e3e2f1333481795a089bef2e24bea/users/f84281091687464990ffbca590b9343b/roles/70c34ea493484322911dab62b4a56e75` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:29.639274 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application [None req-fd5effac-43cc-45a3-a092-280e4f3b906e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 21:59:29.639274 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 21:59:29.639274 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 21:59:29.639274 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 21:59:29.639274 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 21:59:29.639274 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 21:59:29.639274 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 21:59:29.639274 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 21:59:29.639274 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 21:59:29.639274 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 21:59:29.639274 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 21:59:29.639274 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 21:59:29.639274 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 21:59:29.639274 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 21:59:29.639274 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 21:59:29.640265 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 21:59:29.640265 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 21:59:29.640265 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 21:59:29.640265 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 21:59:29.640265 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 21:59:29.640265 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 21:59:29.640265 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 21:59:29.640265 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 21:59:29.640265 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application with Lock( Jul 13 21:59:29.640265 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 21:59:29.640265 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._enter() Jul 13 21:59:29.640265 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 21:59:29.640265 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 21:59:29.640265 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 21:59:29.640265 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.creator() Jul 13 21:59:29.640265 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 21:59:29.640265 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application created_value = creator( Jul 13 21:59:29.641346 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 21:59:29.641346 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 21:59:29.641346 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 21:59:29.641346 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Jul 13 21:59:29.648409 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 288/591] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Jul 13 21:59:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 190 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 21:59:29.648409 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5effac-43cc-45a3-a092-280e4f3b906e None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:29.660112 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4575a604-3529-4cc5-ace6-e962f8c52668 None admin] RBAC: Authorizing `identity:create_grant(project_id=417e3e2f1333481795a089bef2e24bea, user_id=f84281091687464990ffbca590b9343b, role_id=70c34ea493484322911dab62b4a56e75)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:29.662054 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4575a604-3529-4cc5-ace6-e962f8c52668 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:29.676856 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 304/592] 10.222.0.52 () {58 vars in 1439 bytes} [Wed Jul 13 21:59:29 2022] PUT /identity/v3/projects/417e3e2f1333481795a089bef2e24bea/users/f84281091687464990ffbca590b9343b/roles/70c34ea493484322911dab62b4a56e75 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 21:59:29.677884 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4575a604-3529-4cc5-ace6-e962f8c52668 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:29.757927 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:59:29.795382 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd5effac-43cc-45a3-a092-280e4f3b906e None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:29.828950 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4575a604-3529-4cc5-ace6-e962f8c52668 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:29.849999 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be3c5ef-fbd6-453d-ad44-54970de32838 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:29.850811 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be3c5ef-fbd6-453d-ad44-54970de32838 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:29.851163 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be3c5ef-fbd6-453d-ad44-54970de32838 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:29.851499 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be3c5ef-fbd6-453d-ad44-54970de32838 None admin] PATH_INFO: `/v3/domains` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:29.852957 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be3c5ef-fbd6-453d-ad44-54970de32838 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 21:59:29.853341 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be3c5ef-fbd6-453d-ad44-54970de32838 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:29.855085 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be3c5ef-fbd6-453d-ad44-54970de32838 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:29.868225 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 289/593] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Jul 13 21:59:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 225 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 21:59:29.869206 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be3c5ef-fbd6-453d-ad44-54970de32838 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:29.875681 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6138fcf7-1e16-46fc-9bef-f89ccad58a30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:29.876527 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6138fcf7-1e16-46fc-9bef-f89ccad58a30 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:29.876807 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6138fcf7-1e16-46fc-9bef-f89ccad58a30 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:29.877079 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6138fcf7-1e16-46fc-9bef-f89ccad58a30 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:29.886072 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6138fcf7-1e16-46fc-9bef-f89ccad58a30 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:29.887572 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6138fcf7-1e16-46fc-9bef-f89ccad58a30 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:29.887769 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:59:29.903506 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application [None req-6138fcf7-1e16-46fc-9bef-f89ccad58a30 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 13 21:59:29.903506 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 21:59:29.903506 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 21:59:29.903506 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 21:59:29.903506 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 21:59:29.903506 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 21:59:29.903506 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 21:59:29.903506 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 21:59:29.903506 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 21:59:29.903506 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 21:59:29.903506 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 21:59:29.903506 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 21:59:29.903506 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 13 21:59:29.903506 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 13 21:59:29.903506 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 13 21:59:29.903506 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 13 21:59:29.904481 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 21:59:29.904481 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 21:59:29.904481 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 13 21:59:29.904481 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 21:59:29.904481 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 13 21:59:29.904481 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 21:59:29.904481 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 21:59:29.904481 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 21:59:29.904481 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 21:59:29.904481 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 21:59:29.904481 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 21:59:29.904481 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application with Lock( Jul 13 21:59:29.904481 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 21:59:29.904481 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._enter() Jul 13 21:59:29.904481 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 21:59:29.904481 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 21:59:29.904481 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 21:59:29.904481 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.creator() Jul 13 21:59:29.905421 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 21:59:29.905421 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application created_value = creator( Jul 13 21:59:29.905421 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 13 21:59:29.905421 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 13 21:59:29.905421 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 13 21:59:29.905421 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 13 21:59:29.905421 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 13 21:59:29.905421 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 13 21:59:29.905421 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 13 21:59:29.905421 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Jul 13 21:59:29.905421 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 305/594] 10.222.0.52 () {56 vars in 1102 bytes} [Wed Jul 13 21:59:29 2022] GET /identity/v3/users/neutron => generated 85 bytes in 227 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 21:59:29.906014 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6138fcf7-1e16-46fc-9bef-f89ccad58a30 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:29.985935 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be3c5ef-fbd6-453d-ad44-54970de32838 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:30.019816 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db768ae0-5e7e-43f9-9dbd-2b6929ebede9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:30.020536 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db768ae0-5e7e-43f9-9dbd-2b6929ebede9 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:30.020814 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db768ae0-5e7e-43f9-9dbd-2b6929ebede9 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:30.021084 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db768ae0-5e7e-43f9-9dbd-2b6929ebede9 None admin] PATH_INFO: `/v3/domains` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:30.021974 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db768ae0-5e7e-43f9-9dbd-2b6929ebede9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 21:59:30.022265 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6138fcf7-1e16-46fc-9bef-f89ccad58a30 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:30.022387 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db768ae0-5e7e-43f9-9dbd-2b6929ebede9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:30.023803 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db768ae0-5e7e-43f9-9dbd-2b6929ebede9 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:30.036837 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 290/595] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Jul 13 21:59:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 21:59:30.037679 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db768ae0-5e7e-43f9-9dbd-2b6929ebede9 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:30.037810 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:59:30.061315 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd20d5c7-a3a5-4ccd-892b-c8f84417da3d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:30.062127 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd20d5c7-a3a5-4ccd-892b-c8f84417da3d None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:30.062403 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd20d5c7-a3a5-4ccd-892b-c8f84417da3d None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:30.062706 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd20d5c7-a3a5-4ccd-892b-c8f84417da3d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:30.070196 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd20d5c7-a3a5-4ccd-892b-c8f84417da3d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:30.071843 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd20d5c7-a3a5-4ccd-892b-c8f84417da3d None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:30.088593 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application [None req-cd20d5c7-a3a5-4ccd-892b-c8f84417da3d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 13 21:59:30.088593 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 21:59:30.088593 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 21:59:30.088593 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 21:59:30.088593 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 21:59:30.088593 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 21:59:30.088593 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 21:59:30.088593 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 21:59:30.088593 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 21:59:30.088593 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 21:59:30.088593 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 21:59:30.088593 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 21:59:30.088593 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 13 21:59:30.088593 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 13 21:59:30.088593 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 13 21:59:30.089531 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 13 21:59:30.089531 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 21:59:30.089531 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 21:59:30.089531 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 21:59:30.089531 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 21:59:30.089531 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 21:59:30.089531 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 21:59:30.089531 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 21:59:30.089531 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application with Lock( Jul 13 21:59:30.089531 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 21:59:30.089531 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._enter() Jul 13 21:59:30.089531 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 21:59:30.089531 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 21:59:30.089531 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 21:59:30.089531 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.creator() Jul 13 21:59:30.089531 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 21:59:30.089531 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application created_value = creator( Jul 13 21:59:30.090424 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 13 21:59:30.090424 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 13 21:59:30.090424 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 13 21:59:30.090424 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 13 21:59:30.090424 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 13 21:59:30.090424 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 13 21:59:30.090424 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 13 21:59:30.090424 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Jul 13 21:59:30.094597 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 306/596] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Jul 13 21:59:29 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 21:59:30.095472 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd20d5c7-a3a5-4ccd-892b-c8f84417da3d None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:30.123381 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:59:30.145136 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db768ae0-5e7e-43f9-9dbd-2b6929ebede9 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:30.203311 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2890938-ad6f-4016-b096-2e8b196d980f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:30.204083 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2890938-ad6f-4016-b096-2e8b196d980f None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:30.204375 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2890938-ad6f-4016-b096-2e8b196d980f None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:30.204653 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2890938-ad6f-4016-b096-2e8b196d980f None admin] PATH_INFO: `/v3/users` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:30.205517 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2890938-ad6f-4016-b096-2e8b196d980f None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 21:59:30.205826 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2890938-ad6f-4016-b096-2e8b196d980f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:30.207431 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2890938-ad6f-4016-b096-2e8b196d980f None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:30.216336 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd20d5c7-a3a5-4ccd-892b-c8f84417da3d None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:30.226474 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 291/597] 10.222.0.52 () {56 vars in 1139 bytes} [Wed Jul 13 21:59:30 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 21:59:30.227451 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2890938-ad6f-4016-b096-2e8b196d980f None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:30.250356 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2e548c-55a8-4894-9ffd-7664fb9a0466 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:30.251114 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2e548c-55a8-4894-9ffd-7664fb9a0466 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:30.251389 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2e548c-55a8-4894-9ffd-7664fb9a0466 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:30.251661 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c2e548c-55a8-4894-9ffd-7664fb9a0466 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:30.258610 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2e548c-55a8-4894-9ffd-7664fb9a0466 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:30.260151 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c2e548c-55a8-4894-9ffd-7664fb9a0466 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:30.274764 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application [None req-8c2e548c-55a8-4894-9ffd-7664fb9a0466 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 13 21:59:30.274764 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 21:59:30.274764 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 21:59:30.274764 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 21:59:30.274764 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 21:59:30.274764 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 21:59:30.274764 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 21:59:30.274764 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 21:59:30.274764 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 21:59:30.274764 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 21:59:30.274764 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 21:59:30.274764 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 21:59:30.274764 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 13 21:59:30.274764 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 13 21:59:30.274764 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 13 21:59:30.275711 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 13 21:59:30.275711 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 21:59:30.275711 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 21:59:30.275711 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 21:59:30.275711 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 21:59:30.275711 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 21:59:30.275711 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 21:59:30.275711 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 21:59:30.275711 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application with Lock( Jul 13 21:59:30.275711 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 21:59:30.275711 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._enter() Jul 13 21:59:30.275711 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 21:59:30.275711 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 21:59:30.275711 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 21:59:30.275711 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.creator() Jul 13 21:59:30.275711 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 21:59:30.275711 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application created_value = creator( Jul 13 21:59:30.276720 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 13 21:59:30.276720 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 13 21:59:30.276720 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 13 21:59:30.276720 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 13 21:59:30.276720 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 13 21:59:30.276720 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 13 21:59:30.276720 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 13 21:59:30.276720 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Jul 13 21:59:30.276720 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 307/598] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Jul 13 21:59:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 21:59:30.277205 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2e548c-55a8-4894-9ffd-7664fb9a0466 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:30.316503 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:59:30.343044 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2890938-ad6f-4016-b096-2e8b196d980f None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:30.376769 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:59:30.383913 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696973f8-d16f-4960-841c-4ca7a84b6b52 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:30.384139 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c2e548c-55a8-4894-9ffd-7664fb9a0466 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:30.385711 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696973f8-d16f-4960-841c-4ca7a84b6b52 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:30.386095 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696973f8-d16f-4960-841c-4ca7a84b6b52 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:30.386608 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-696973f8-d16f-4960-841c-4ca7a84b6b52 None admin] PATH_INFO: `/v3/projects` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:30.387516 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696973f8-d16f-4960-841c-4ca7a84b6b52 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 21:59:30.387927 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696973f8-d16f-4960-841c-4ca7a84b6b52 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:30.389478 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-696973f8-d16f-4960-841c-4ca7a84b6b52 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:30.408523 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 292/599] 10.222.0.52 () {56 vars in 1148 bytes} [Wed Jul 13 21:59:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 21:59:30.409671 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696973f8-d16f-4960-841c-4ca7a84b6b52 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:30.420023 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12822fb-653a-4dbf-a6a5-7868725ec651 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:30.420836 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12822fb-653a-4dbf-a6a5-7868725ec651 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:30.421109 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12822fb-653a-4dbf-a6a5-7868725ec651 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:30.421377 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c12822fb-653a-4dbf-a6a5-7868725ec651 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:30.432032 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c12822fb-653a-4dbf-a6a5-7868725ec651 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:30.433486 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c12822fb-653a-4dbf-a6a5-7868725ec651 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:30.451321 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application [None req-c12822fb-653a-4dbf-a6a5-7868725ec651 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 21:59:30.451321 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 21:59:30.451321 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 21:59:30.451321 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 21:59:30.451321 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 21:59:30.451321 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 21:59:30.451321 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 21:59:30.451321 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 21:59:30.451321 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 21:59:30.451321 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 21:59:30.451321 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 21:59:30.451321 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 21:59:30.451321 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 21:59:30.451321 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 21:59:30.451321 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 21:59:30.452402 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 21:59:30.452402 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 21:59:30.452402 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 21:59:30.452402 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 21:59:30.452402 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 21:59:30.452402 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 21:59:30.452402 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 21:59:30.452402 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 21:59:30.452402 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application with Lock( Jul 13 21:59:30.452402 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 21:59:30.452402 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._enter() Jul 13 21:59:30.452402 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 21:59:30.452402 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 21:59:30.452402 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 21:59:30.452402 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.creator() Jul 13 21:59:30.452402 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 21:59:30.452402 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application created_value = creator( Jul 13 21:59:30.453338 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 21:59:30.453338 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 21:59:30.453338 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 21:59:30.453338 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Jul 13 21:59:30.453338 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 308/600] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Jul 13 21:59:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 21:59:30.453621 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12822fb-653a-4dbf-a6a5-7868725ec651 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:30.528574 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-696973f8-d16f-4960-841c-4ca7a84b6b52 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:30.559618 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:59:30.565570 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349638a4-c19f-4bbf-8f9b-2375b9dc8c7b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:30.566340 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349638a4-c19f-4bbf-8f9b-2375b9dc8c7b None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:30.566669 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349638a4-c19f-4bbf-8f9b-2375b9dc8c7b None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:30.566944 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349638a4-c19f-4bbf-8f9b-2375b9dc8c7b None admin] PATH_INFO: `/v3/projects` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:30.567727 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-349638a4-c19f-4bbf-8f9b-2375b9dc8c7b None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 21:59:30.568183 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-349638a4-c19f-4bbf-8f9b-2375b9dc8c7b None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:30.570452 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-349638a4-c19f-4bbf-8f9b-2375b9dc8c7b None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:30.578429 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c12822fb-653a-4dbf-a6a5-7868725ec651 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:30.589273 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 293/601] 10.222.0.52 () {56 vars in 1148 bytes} [Wed Jul 13 21:59:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 21:59:30.590413 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349638a4-c19f-4bbf-8f9b-2375b9dc8c7b None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:30.615366 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d229e8-7d6c-4476-a33d-4ca99f18e06e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:30.616596 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d229e8-7d6c-4476-a33d-4ca99f18e06e None admin] REQUEST_METHOD: `PUT` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:30.617052 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d229e8-7d6c-4476-a33d-4ca99f18e06e None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:30.617497 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2d229e8-7d6c-4476-a33d-4ca99f18e06e None admin] PATH_INFO: `/v3/projects/417e3e2f1333481795a089bef2e24bea/users/e4d35d50e15d43d3afc59206b8165f30/roles/70c34ea493484322911dab62b4a56e75` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:30.633519 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d229e8-7d6c-4476-a33d-4ca99f18e06e None admin] RBAC: Authorizing `identity:create_grant(project_id=417e3e2f1333481795a089bef2e24bea, user_id=e4d35d50e15d43d3afc59206b8165f30, role_id=70c34ea493484322911dab62b4a56e75)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:30.636057 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2d229e8-7d6c-4476-a33d-4ca99f18e06e None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:30.640699 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:59:30.651709 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 309/602] 10.222.0.52 () {58 vars in 1439 bytes} [Wed Jul 13 21:59:30 2022] PUT /identity/v3/projects/417e3e2f1333481795a089bef2e24bea/users/e4d35d50e15d43d3afc59206b8165f30/roles/70c34ea493484322911dab62b4a56e75 => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 21:59:30.652811 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d229e8-7d6c-4476-a33d-4ca99f18e06e None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:30.715773 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349638a4-c19f-4bbf-8f9b-2375b9dc8c7b None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:30.765979 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d601878-bc7d-4967-bd47-70d71663af8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:30.767215 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d601878-bc7d-4967-bd47-70d71663af8a None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:30.767636 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d601878-bc7d-4967-bd47-70d71663af8a None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:30.768221 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d601878-bc7d-4967-bd47-70d71663af8a None admin] PATH_INFO: `/v3/domains` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:30.769431 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d601878-bc7d-4967-bd47-70d71663af8a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 21:59:30.769881 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d601878-bc7d-4967-bd47-70d71663af8a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:30.771476 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d601878-bc7d-4967-bd47-70d71663af8a None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:30.785899 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2d229e8-7d6c-4476-a33d-4ca99f18e06e None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:30.789476 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 294/603] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Jul 13 21:59:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 21:59:30.791503 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cdd2e1-0f0b-4320-991b-e33d111fc092 None None] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:30.791935 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cdd2e1-0f0b-4320-991b-e33d111fc092 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:30.792371 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06cdd2e1-0f0b-4320-991b-e33d111fc092 None None] PATH_INFO: `/` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:30.798649 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 295/604] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jul 13 21:59:30 2022] GET /identity => generated 270 bytes in 8 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 21:59:30.799783 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cdd2e1-0f0b-4320-991b-e33d111fc092 None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:30.834184 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30cdbc71-2ecc-4fc3-99f2-f59122db4dcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:30.837561 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cdbc71-2ecc-4fc3-99f2-f59122db4dcf None admin] REQUEST_METHOD: `PUT` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:30.837561 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cdbc71-2ecc-4fc3-99f2-f59122db4dcf None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:30.837561 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30cdbc71-2ecc-4fc3-99f2-f59122db4dcf None admin] PATH_INFO: `/v3/projects/417e3e2f1333481795a089bef2e24bea/users/bc77e1ea1ada4700b809beb1c980c5e1/roles/70c34ea493484322911dab62b4a56e75` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:30.842283 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:59:30.857216 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30cdbc71-2ecc-4fc3-99f2-f59122db4dcf None admin] RBAC: Authorizing `identity:create_grant(project_id=417e3e2f1333481795a089bef2e24bea, user_id=bc77e1ea1ada4700b809beb1c980c5e1, role_id=70c34ea493484322911dab62b4a56e75)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:30.861761 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30cdbc71-2ecc-4fc3-99f2-f59122db4dcf None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:30.877770 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 310/605] 10.222.0.52 () {58 vars in 1439 bytes} [Wed Jul 13 21:59:30 2022] PUT /identity/v3/projects/417e3e2f1333481795a089bef2e24bea/users/bc77e1ea1ada4700b809beb1c980c5e1/roles/70c34ea493484322911dab62b4a56e75 => generated 0 bytes in 225 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 21:59:30.879268 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61ede68-1b7d-4416-85bf-65acbba063b4 None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:30.879552 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61ede68-1b7d-4416-85bf-65acbba063b4 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:30.879827 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b61ede68-1b7d-4416-85bf-65acbba063b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:30.899233 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-b61ede68-1b7d-4416-85bf-65acbba063b4 None None] MFA Rules not processed for user `cc99c93da5074cf5a48bc2aa9a92c5e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:59:30.901206 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-b61ede68-1b7d-4416-85bf-65acbba063b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:59:31.002887 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 311/606] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jul 13 21:59:30 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 13 21:59:31.017688 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06cdd2e1-0f0b-4320-991b-e33d111fc092 None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:31.026107 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61ede68-1b7d-4416-85bf-65acbba063b4 None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:31.029725 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:59:31.095196 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079ea847-a645-470b-b930-5ce5a755f9cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:31.095959 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079ea847-a645-470b-b930-5ce5a755f9cb None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:31.096230 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079ea847-a645-470b-b930-5ce5a755f9cb None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:31.096495 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079ea847-a645-470b-b930-5ce5a755f9cb None admin] PATH_INFO: `/v3/projects/service` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:31.109520 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-079ea847-a645-470b-b930-5ce5a755f9cb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:31.112604 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-079ea847-a645-470b-b930-5ce5a755f9cb None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:31.131181 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application [None req-079ea847-a645-470b-b930-5ce5a755f9cb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 13 21:59:31.131181 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 21:59:31.131181 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 21:59:31.131181 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 21:59:31.131181 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 21:59:31.131181 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 21:59:31.131181 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 21:59:31.131181 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 21:59:31.131181 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 21:59:31.131181 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 21:59:31.131181 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 21:59:31.131181 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 21:59:31.131181 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 13 21:59:31.131181 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 13 21:59:31.131181 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 13 21:59:31.132128 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 13 21:59:31.132128 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 21:59:31.132128 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 21:59:31.132128 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 21:59:31.132128 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 21:59:31.132128 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 21:59:31.132128 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 21:59:31.132128 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 21:59:31.132128 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application with Lock( Jul 13 21:59:31.132128 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 21:59:31.132128 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._enter() Jul 13 21:59:31.132128 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 21:59:31.132128 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 21:59:31.132128 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 21:59:31.132128 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.creator() Jul 13 21:59:31.132128 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 21:59:31.132128 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application created_value = creator( Jul 13 21:59:31.133025 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 13 21:59:31.133025 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 13 21:59:31.133025 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 13 21:59:31.133025 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 13 21:59:31.133025 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 13 21:59:31.133025 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 13 21:59:31.133025 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 13 21:59:31.133025 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Jul 13 21:59:31.133453 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 296/607] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Jul 13 21:59:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 334 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 21:59:31.140809 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079ea847-a645-470b-b930-5ce5a755f9cb None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:31.185295 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b61ede68-1b7d-4416-85bf-65acbba063b4 None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:31.210953 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:59:31.248307 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8bb79d-22a7-4a62-95bb-7c8fa3d5172f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:31.249502 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8bb79d-22a7-4a62-95bb-7c8fa3d5172f None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:31.249778 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8bb79d-22a7-4a62-95bb-7c8fa3d5172f None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:31.250045 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c8bb79d-22a7-4a62-95bb-7c8fa3d5172f None admin] PATH_INFO: `/v3/roles/service` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:31.277231 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c8bb79d-22a7-4a62-95bb-7c8fa3d5172f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 21:59:31.290101 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application [None req-5c8bb79d-22a7-4a62-95bb-7c8fa3d5172f None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 13 21:59:31.290101 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 21:59:31.290101 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 21:59:31.290101 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 21:59:31.290101 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 21:59:31.290101 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 21:59:31.290101 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 21:59:31.290101 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 21:59:31.290101 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 21:59:31.290101 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 21:59:31.290101 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 21:59:31.290101 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 21:59:31.290101 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 21:59:31.290101 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 21:59:31.290101 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 21:59:31.290101 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 21:59:31.291649 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 21:59:31.291649 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 21:59:31.291649 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 21:59:31.291649 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 21:59:31.291649 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 21:59:31.291649 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 21:59:31.291649 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 21:59:31.291649 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 21:59:31.291649 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 21:59:31.291649 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 21:59:31.291649 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 21:59:31.291649 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application with Lock( Jul 13 21:59:31.291649 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 21:59:31.291649 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._enter() Jul 13 21:59:31.291649 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 21:59:31.291649 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 21:59:31.292487 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 21:59:31.292487 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.creator() Jul 13 21:59:31.292487 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 21:59:31.292487 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application created_value = creator( Jul 13 21:59:31.292487 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 21:59:31.292487 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 21:59:31.292487 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 21:59:31.292487 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 21:59:31.292487 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 21:59:31.292487 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 21:59:31.292487 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 13 21:59:31.292487 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Jul 13 21:59:31.295903 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079ea847-a645-470b-b930-5ce5a755f9cb None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:31.296377 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 312/608] 10.222.0.52 () {56 vars in 1102 bytes} [Wed Jul 13 21:59:31 2022] GET /identity/v3/roles/service => generated 85 bytes in 272 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 13 21:59:31.305376 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8bb79d-22a7-4a62-95bb-7c8fa3d5172f None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:31.367221 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c573ec4-7383-4b9c-8369-28ce7146d31c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:31.397860 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c573ec4-7383-4b9c-8369-28ce7146d31c None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:31.397860 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c573ec4-7383-4b9c-8369-28ce7146d31c None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:31.397860 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c573ec4-7383-4b9c-8369-28ce7146d31c None admin] PATH_INFO: `/v3/projects` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:31.397860 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c573ec4-7383-4b9c-8369-28ce7146d31c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 21:59:31.397860 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c573ec4-7383-4b9c-8369-28ce7146d31c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:31.397860 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c573ec4-7383-4b9c-8369-28ce7146d31c None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:31.398322 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 297/609] 10.222.0.52 () {56 vars in 1148 bytes} [Wed Jul 13 21:59:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 257 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 21:59:31.404547 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c573ec4-7383-4b9c-8369-28ce7146d31c None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:31.457174 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:59:31.485749 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c8bb79d-22a7-4a62-95bb-7c8fa3d5172f None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:31.520479 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac0994d-c4df-4b6b-a7ca-1e2ba1e0e924 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:31.521300 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac0994d-c4df-4b6b-a7ca-1e2ba1e0e924 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:31.521586 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac0994d-c4df-4b6b-a7ca-1e2ba1e0e924 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:31.521850 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bac0994d-c4df-4b6b-a7ca-1e2ba1e0e924 None admin] PATH_INFO: `/v3/roles` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:31.522641 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac0994d-c4df-4b6b-a7ca-1e2ba1e0e924 None admin] RBAC: Adding query filter params (name=service) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 21:59:31.522942 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac0994d-c4df-4b6b-a7ca-1e2ba1e0e924 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:31.524329 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bac0994d-c4df-4b6b-a7ca-1e2ba1e0e924 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:31.534113 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 313/610] 10.222.0.52 () {56 vars in 1103 bytes} [Wed Jul 13 21:59:31 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 21:59:31.543576 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac0994d-c4df-4b6b-a7ca-1e2ba1e0e924 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:31.590611 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c573ec4-7383-4b9c-8369-28ce7146d31c None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:31.616605 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:59:31.660793 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5591890-0f7a-4566-9bf8-6cfd85f6ee65 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:31.661695 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5591890-0f7a-4566-9bf8-6cfd85f6ee65 None admin] REQUEST_METHOD: `PUT` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:31.661973 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5591890-0f7a-4566-9bf8-6cfd85f6ee65 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:31.662245 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5591890-0f7a-4566-9bf8-6cfd85f6ee65 None admin] PATH_INFO: `/v3/projects/417e3e2f1333481795a089bef2e24bea/users/847ec9a1bd4d4dec804318a5c845895a/roles/70c34ea493484322911dab62b4a56e75` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:31.682736 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5591890-0f7a-4566-9bf8-6cfd85f6ee65 None admin] RBAC: Authorizing `identity:create_grant(project_id=417e3e2f1333481795a089bef2e24bea, user_id=847ec9a1bd4d4dec804318a5c845895a, role_id=70c34ea493484322911dab62b4a56e75)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:31.684207 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5591890-0f7a-4566-9bf8-6cfd85f6ee65 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:31.702708 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 298/611] 10.222.0.52 () {58 vars in 1439 bytes} [Wed Jul 13 21:59:31 2022] PUT /identity/v3/projects/417e3e2f1333481795a089bef2e24bea/users/847ec9a1bd4d4dec804318a5c845895a/roles/70c34ea493484322911dab62b4a56e75 => generated 0 bytes in 300 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 13 21:59:31.720825 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bac0994d-c4df-4b6b-a7ca-1e2ba1e0e924 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:31.757299 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ea634f-0774-4581-be69-296064a9d85d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:31.758168 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ea634f-0774-4581-be69-296064a9d85d None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:31.758521 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ea634f-0774-4581-be69-296064a9d85d None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:31.758798 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ea634f-0774-4581-be69-296064a9d85d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:31.770722 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ea634f-0774-4581-be69-296064a9d85d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:31.772213 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ea634f-0774-4581-be69-296064a9d85d None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:31.792375 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application [None req-64ea634f-0774-4581-be69-296064a9d85d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 21:59:31.792375 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 21:59:31.792375 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 21:59:31.792375 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 21:59:31.792375 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 21:59:31.792375 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 21:59:31.792375 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 21:59:31.792375 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 21:59:31.792375 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 21:59:31.792375 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 21:59:31.792375 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 21:59:31.792375 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 21:59:31.792375 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 21:59:31.792375 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 21:59:31.792375 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 21:59:31.793377 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 21:59:31.793377 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 21:59:31.793377 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 21:59:31.793377 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 21:59:31.793377 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 21:59:31.793377 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 21:59:31.793377 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 21:59:31.793377 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 21:59:31.793377 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application with Lock( Jul 13 21:59:31.793377 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 21:59:31.793377 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._enter() Jul 13 21:59:31.793377 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 21:59:31.793377 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 21:59:31.793377 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 21:59:31.793377 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.creator() Jul 13 21:59:31.793377 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 21:59:31.793377 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application created_value = creator( Jul 13 21:59:31.794301 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 21:59:31.794301 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 21:59:31.794301 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 21:59:31.794301 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Jul 13 21:59:31.794301 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 314/612] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Jul 13 21:59:31 2022] GET /identity/v3/domains/Default => generated 87 bytes in 251 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 21:59:31.801008 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5591890-0f7a-4566-9bf8-6cfd85f6ee65 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:31.886863 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:59:31.911647 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5591890-0f7a-4566-9bf8-6cfd85f6ee65 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:31.951866 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93dc70b9-fb30-457c-a1bc-f902083d9c4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:31.952604 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93dc70b9-fb30-457c-a1bc-f902083d9c4e None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:31.952884 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93dc70b9-fb30-457c-a1bc-f902083d9c4e None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:31.953205 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93dc70b9-fb30-457c-a1bc-f902083d9c4e None admin] PATH_INFO: `/v3/domains` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:31.953962 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93dc70b9-fb30-457c-a1bc-f902083d9c4e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 21:59:31.954267 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93dc70b9-fb30-457c-a1bc-f902083d9c4e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:31.955812 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93dc70b9-fb30-457c-a1bc-f902083d9c4e None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:31.971162 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 299/613] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Jul 13 21:59:31 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 21:59:31.977490 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ea634f-0774-4581-be69-296064a9d85d None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:32.071006 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:59:32.080441 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3fb16e-968f-4ea6-ac27-fbe8773c58e9 None None] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:32.080723 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3fb16e-968f-4ea6-ac27-fbe8773c58e9 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:32.080999 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f3fb16e-968f-4ea6-ac27-fbe8773c58e9 None None] PATH_INFO: `/` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:32.082634 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 300/614] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jul 13 21:59:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 21:59:32.084789 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ea634f-0774-4581-be69-296064a9d85d None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:32.092310 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb20110-a735-4ea5-b7b8-6ebd13af5f79 None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:32.092310 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb20110-a735-4ea5-b7b8-6ebd13af5f79 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:32.092310 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6bb20110-a735-4ea5-b7b8-6ebd13af5f79 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:32.121957 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862b6255-c5b8-4847-a623-88411fe3671c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:32.122922 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862b6255-c5b8-4847-a623-88411fe3671c None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:32.123206 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862b6255-c5b8-4847-a623-88411fe3671c None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:32.123479 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-862b6255-c5b8-4847-a623-88411fe3671c None admin] PATH_INFO: `/v3/users/swift` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:32.130778 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-862b6255-c5b8-4847-a623-88411fe3671c None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:32.131783 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-6bb20110-a735-4ea5-b7b8-6ebd13af5f79 None None] MFA Rules not processed for user `cc99c93da5074cf5a48bc2aa9a92c5e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:59:32.132219 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-862b6255-c5b8-4847-a623-88411fe3671c None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:32.133417 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-6bb20110-a735-4ea5-b7b8-6ebd13af5f79 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:59:32.148336 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application [None req-862b6255-c5b8-4847-a623-88411fe3671c None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 13 21:59:32.148336 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 21:59:32.148336 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 21:59:32.148336 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 21:59:32.148336 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 21:59:32.148336 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 21:59:32.148336 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 21:59:32.148336 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 21:59:32.148336 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 21:59:32.148336 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 21:59:32.148336 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 21:59:32.148336 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 21:59:32.148336 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 13 21:59:32.148336 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 13 21:59:32.148336 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 13 21:59:32.148336 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 13 21:59:32.149318 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 21:59:32.149318 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 21:59:32.149318 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 13 21:59:32.149318 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 21:59:32.149318 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 13 21:59:32.149318 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 21:59:32.149318 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 21:59:32.149318 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 21:59:32.149318 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 21:59:32.149318 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 21:59:32.149318 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 21:59:32.149318 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application with Lock( Jul 13 21:59:32.149318 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 21:59:32.149318 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._enter() Jul 13 21:59:32.149318 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 21:59:32.149318 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 21:59:32.149318 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 21:59:32.149318 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.creator() Jul 13 21:59:32.150308 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 21:59:32.150308 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application created_value = creator( Jul 13 21:59:32.150308 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 13 21:59:32.150308 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 13 21:59:32.150308 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 13 21:59:32.150308 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 13 21:59:32.150308 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 13 21:59:32.150308 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 13 21:59:32.150308 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 13 21:59:32.150308 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Jul 13 21:59:32.150308 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 315/615] 10.222.0.52 () {56 vars in 1096 bytes} [Wed Jul 13 21:59:31 2022] GET /identity/v3/users/swift => generated 83 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 21:59:32.158517 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862b6255-c5b8-4847-a623-88411fe3671c None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:32.263978 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-862b6255-c5b8-4847-a623-88411fe3671c None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:32.307037 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 301/616] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jul 13 21:59:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 217 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 13 21:59:32.326254 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3827fe29-7c91-4a0e-8387-8534fcc4a78f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:32.326963 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3827fe29-7c91-4a0e-8387-8534fcc4a78f None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:32.327247 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3827fe29-7c91-4a0e-8387-8534fcc4a78f None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:32.327516 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3827fe29-7c91-4a0e-8387-8534fcc4a78f None admin] PATH_INFO: `/v3/users` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:32.328375 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3827fe29-7c91-4a0e-8387-8534fcc4a78f None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 21:59:32.328683 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3827fe29-7c91-4a0e-8387-8534fcc4a78f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:32.330219 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3827fe29-7c91-4a0e-8387-8534fcc4a78f None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:32.339491 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb20110-a735-4ea5-b7b8-6ebd13af5f79 None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:32.351564 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 316/617] 10.222.0.52 () {56 vars in 1135 bytes} [Wed Jul 13 21:59:32 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 21:59:32.357571 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3827fe29-7c91-4a0e-8387-8534fcc4a78f None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:32.382521 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:59:32.469704 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3827fe29-7c91-4a0e-8387-8534fcc4a78f None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:32.506913 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3dfa538-25df-4b9c-b150-9a16cb8fd6e3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:32.508268 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3dfa538-25df-4b9c-b150-9a16cb8fd6e3 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:32.508646 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3dfa538-25df-4b9c-b150-9a16cb8fd6e3 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:32.509125 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3dfa538-25df-4b9c-b150-9a16cb8fd6e3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:32.513320 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:59:32.521908 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3dfa538-25df-4b9c-b150-9a16cb8fd6e3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:32.523527 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3dfa538-25df-4b9c-b150-9a16cb8fd6e3 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:32.556029 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application [None req-e3dfa538-25df-4b9c-b150-9a16cb8fd6e3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 21:59:32.556029 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 21:59:32.556029 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 21:59:32.556029 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 21:59:32.556029 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 21:59:32.556029 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 21:59:32.556029 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 21:59:32.556029 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 21:59:32.556029 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 21:59:32.556029 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 21:59:32.556029 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 21:59:32.556029 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 21:59:32.556029 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 21:59:32.556029 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 21:59:32.556029 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 21:59:32.557551 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 21:59:32.557551 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 21:59:32.557551 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 21:59:32.557551 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 21:59:32.557551 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 21:59:32.557551 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 21:59:32.557551 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 21:59:32.557551 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 21:59:32.557551 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application with Lock( Jul 13 21:59:32.557551 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 21:59:32.557551 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._enter() Jul 13 21:59:32.557551 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 21:59:32.557551 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 21:59:32.557551 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 21:59:32.557551 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.creator() Jul 13 21:59:32.557551 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 21:59:32.557551 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application created_value = creator( Jul 13 21:59:32.558472 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 21:59:32.558472 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 21:59:32.558472 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 21:59:32.558472 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Jul 13 21:59:32.560768 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 317/618] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Jul 13 21:59:32 2022] GET /identity/v3/domains/Default => generated 87 bytes in 203 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 21:59:32.567292 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3dfa538-25df-4b9c-b150-9a16cb8fd6e3 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:32.584673 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6bb20110-a735-4ea5-b7b8-6ebd13af5f79 None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:32.640371 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1755c56-b4e3-497f-885b-7d024b0ec528 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:32.641570 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1755c56-b4e3-497f-885b-7d024b0ec528 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:32.642078 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1755c56-b4e3-497f-885b-7d024b0ec528 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:32.642576 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1755c56-b4e3-497f-885b-7d024b0ec528 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:32.655221 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1755c56-b4e3-497f-885b-7d024b0ec528 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 21:59:32.663865 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application [None req-b1755c56-b4e3-497f-885b-7d024b0ec528 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 13 21:59:32.663865 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 21:59:32.663865 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 21:59:32.663865 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 21:59:32.663865 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 21:59:32.663865 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 21:59:32.663865 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 21:59:32.663865 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 21:59:32.663865 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 21:59:32.663865 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 21:59:32.663865 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 21:59:32.663865 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 21:59:32.663865 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 21:59:32.663865 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 21:59:32.663865 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 21:59:32.663865 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 21:59:32.664892 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 21:59:32.664892 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 21:59:32.664892 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 21:59:32.664892 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 21:59:32.664892 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 21:59:32.664892 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 21:59:32.664892 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 21:59:32.664892 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 21:59:32.664892 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 21:59:32.664892 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 21:59:32.664892 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 21:59:32.664892 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application with Lock( Jul 13 21:59:32.664892 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 21:59:32.664892 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._enter() Jul 13 21:59:32.664892 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 21:59:32.664892 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 21:59:32.666153 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 21:59:32.666153 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.creator() Jul 13 21:59:32.666153 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 21:59:32.666153 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application created_value = creator( Jul 13 21:59:32.666153 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 21:59:32.666153 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 21:59:32.666153 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 21:59:32.666153 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 21:59:32.666153 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 21:59:32.666153 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 21:59:32.666153 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 13 21:59:32.666153 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Jul 13 21:59:32.669177 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 302/619] 10.222.0.52 () {56 vars in 1102 bytes} [Wed Jul 13 21:59:32 2022] GET /identity/v3/roles/service => generated 85 bytes in 330 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 21:59:32.675064 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1755c56-b4e3-497f-885b-7d024b0ec528 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:32.699913 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:59:32.745087 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3dfa538-25df-4b9c-b150-9a16cb8fd6e3 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:32.782398 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25388c7-765c-456a-a7ce-9e713b7b2e2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:32.783477 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25388c7-765c-456a-a7ce-9e713b7b2e2a None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:32.783982 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25388c7-765c-456a-a7ce-9e713b7b2e2a None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:32.784508 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25388c7-765c-456a-a7ce-9e713b7b2e2a None admin] PATH_INFO: `/v3/domains` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:32.785522 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c25388c7-765c-456a-a7ce-9e713b7b2e2a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 21:59:32.786030 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c25388c7-765c-456a-a7ce-9e713b7b2e2a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:32.787660 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c25388c7-765c-456a-a7ce-9e713b7b2e2a None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:32.801296 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 318/620] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Jul 13 21:59:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 234 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 21:59:32.807825 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25388c7-765c-456a-a7ce-9e713b7b2e2a None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:32.844641 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1755c56-b4e3-497f-885b-7d024b0ec528 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:32.890004 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:59:32.905931 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82a8968-319b-4ea8-98ce-f0b64e598fee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:32.906701 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82a8968-319b-4ea8-98ce-f0b64e598fee None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:32.906988 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82a8968-319b-4ea8-98ce-f0b64e598fee None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:32.907283 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82a8968-319b-4ea8-98ce-f0b64e598fee None admin] PATH_INFO: `/v3/roles` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:32.908026 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82a8968-319b-4ea8-98ce-f0b64e598fee None admin] RBAC: Adding query filter params (name=service) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 21:59:32.908332 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82a8968-319b-4ea8-98ce-f0b64e598fee None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:32.909719 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82a8968-319b-4ea8-98ce-f0b64e598fee None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:32.922515 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 303/621] 10.222.0.52 () {56 vars in 1103 bytes} [Wed Jul 13 21:59:32 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 244 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 21:59:32.922515 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a56fde4-3af8-4d10-90dd-8e255aa35ae8 None None] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:32.922515 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a56fde4-3af8-4d10-90dd-8e255aa35ae8 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:32.922515 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a56fde4-3af8-4d10-90dd-8e255aa35ae8 None None] PATH_INFO: `/` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:32.922515 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 304/622] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jul 13 21:59:32 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 21:59:32.926036 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25388c7-765c-456a-a7ce-9e713b7b2e2a None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:32.927887 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd363c9a-f43c-40c0-ba83-2badccb81240 None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:32.928174 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd363c9a-f43c-40c0-ba83-2badccb81240 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:32.928452 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd363c9a-f43c-40c0-ba83-2badccb81240 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:32.946168 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-fd363c9a-f43c-40c0-ba83-2badccb81240 None None] MFA Rules not processed for user `cc99c93da5074cf5a48bc2aa9a92c5e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:59:32.947564 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-fd363c9a-f43c-40c0-ba83-2badccb81240 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:59:32.950165 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:59:32.969573 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf43fd65-003b-4729-8802-9e8b04f205b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:32.971028 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf43fd65-003b-4729-8802-9e8b04f205b9 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:32.971528 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf43fd65-003b-4729-8802-9e8b04f205b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:32.971950 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf43fd65-003b-4729-8802-9e8b04f205b9 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:32.981033 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf43fd65-003b-4729-8802-9e8b04f205b9 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:32.983348 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf43fd65-003b-4729-8802-9e8b04f205b9 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:33.001022 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application [None req-bf43fd65-003b-4729-8802-9e8b04f205b9 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 13 21:59:33.001022 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 21:59:33.001022 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 21:59:33.001022 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 21:59:33.001022 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 21:59:33.001022 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 21:59:33.001022 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 21:59:33.001022 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 21:59:33.001022 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 21:59:33.001022 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 21:59:33.001022 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 21:59:33.001022 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 21:59:33.001022 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 13 21:59:33.001022 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 13 21:59:33.001022 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 13 21:59:33.002167 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 13 21:59:33.002167 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 21:59:33.002167 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 21:59:33.002167 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 21:59:33.002167 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 21:59:33.002167 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 21:59:33.002167 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 21:59:33.002167 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 21:59:33.002167 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application with Lock( Jul 13 21:59:33.002167 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 21:59:33.002167 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._enter() Jul 13 21:59:33.002167 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 21:59:33.002167 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 21:59:33.002167 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 21:59:33.002167 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.creator() Jul 13 21:59:33.002167 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 21:59:33.002167 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application created_value = creator( Jul 13 21:59:33.003252 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 13 21:59:33.003252 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 13 21:59:33.003252 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 13 21:59:33.003252 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 13 21:59:33.003252 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 13 21:59:33.003252 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 13 21:59:33.003252 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 13 21:59:33.003252 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Jul 13 21:59:33.005485 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 319/623] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Jul 13 21:59:32 2022] GET /identity/v3/projects/service => generated 88 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 21:59:33.006552 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf43fd65-003b-4729-8802-9e8b04f205b9 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:33.046591 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 305/624] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jul 13 21:59:32 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 13 21:59:33.046591 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd363c9a-f43c-40c0-ba83-2badccb81240 None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:33.137518 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf43fd65-003b-4729-8802-9e8b04f205b9 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:33.148678 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd363c9a-f43c-40c0-ba83-2badccb81240 None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:33.167916 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:59:33.177455 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90d813f-50e4-41c5-97be-434bc1971eb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:33.178270 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90d813f-50e4-41c5-97be-434bc1971eb7 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:33.178623 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90d813f-50e4-41c5-97be-434bc1971eb7 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:33.178908 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c90d813f-50e4-41c5-97be-434bc1971eb7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:33.186000 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d7e85e-9c8f-4ecc-9f25-c2ba99ea2ee9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:33.186725 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d7e85e-9c8f-4ecc-9f25-c2ba99ea2ee9 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:33.187002 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d7e85e-9c8f-4ecc-9f25-c2ba99ea2ee9 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:33.187269 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6d7e85e-9c8f-4ecc-9f25-c2ba99ea2ee9 None admin] PATH_INFO: `/v3/projects` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:33.188063 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d7e85e-9c8f-4ecc-9f25-c2ba99ea2ee9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 21:59:33.188368 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d7e85e-9c8f-4ecc-9f25-c2ba99ea2ee9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:33.189797 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6d7e85e-9c8f-4ecc-9f25-c2ba99ea2ee9 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:33.191429 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90d813f-50e4-41c5-97be-434bc1971eb7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:33.193089 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c90d813f-50e4-41c5-97be-434bc1971eb7 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:33.202864 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 306/625] 10.222.0.52 () {56 vars in 1148 bytes} [Wed Jul 13 21:59:33 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 21:59:33.203755 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d7e85e-9c8f-4ecc-9f25-c2ba99ea2ee9 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:33.220722 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application [None req-c90d813f-50e4-41c5-97be-434bc1971eb7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 21:59:33.220722 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 21:59:33.220722 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 21:59:33.220722 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 21:59:33.220722 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 21:59:33.220722 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 21:59:33.220722 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 21:59:33.220722 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 21:59:33.220722 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 21:59:33.220722 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 21:59:33.220722 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 21:59:33.220722 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 21:59:33.220722 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 21:59:33.220722 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 21:59:33.220722 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 21:59:33.221714 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 21:59:33.221714 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 21:59:33.221714 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 21:59:33.221714 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 21:59:33.221714 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 21:59:33.221714 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 21:59:33.221714 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 21:59:33.221714 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 21:59:33.221714 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application with Lock( Jul 13 21:59:33.221714 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 21:59:33.221714 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._enter() Jul 13 21:59:33.221714 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 21:59:33.221714 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 21:59:33.221714 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 21:59:33.221714 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.creator() Jul 13 21:59:33.221714 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 21:59:33.221714 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application created_value = creator( Jul 13 21:59:33.222666 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 21:59:33.222666 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 21:59:33.222666 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 21:59:33.222666 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Jul 13 21:59:33.223397 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 320/626] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Jul 13 21:59:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 21:59:33.224535 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90d813f-50e4-41c5-97be-434bc1971eb7 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:33.267567 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:59:33.351626 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6d7e85e-9c8f-4ecc-9f25-c2ba99ea2ee9 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:33.355541 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c90d813f-50e4-41c5-97be-434bc1971eb7 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:33.396512 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f17aad7-f9cb-429a-8a10-d93bbd2064f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:33.397439 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f17aad7-f9cb-429a-8a10-d93bbd2064f5 None admin] REQUEST_METHOD: `PUT` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:33.397696 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f17aad7-f9cb-429a-8a10-d93bbd2064f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:33.397968 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f17aad7-f9cb-429a-8a10-d93bbd2064f5 None admin] PATH_INFO: `/v3/projects/417e3e2f1333481795a089bef2e24bea/users/1eb1525233ba48c99b15d4cb75db4822/roles/70c34ea493484322911dab62b4a56e75` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:33.400588 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab01278-02b3-436f-abea-6a01c65ee05c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:33.402553 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab01278-02b3-436f-abea-6a01c65ee05c None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:33.403004 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab01278-02b3-436f-abea-6a01c65ee05c None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:33.403444 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ab01278-02b3-436f-abea-6a01c65ee05c None admin] PATH_INFO: `/v3/roles/service` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:33.414592 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f17aad7-f9cb-429a-8a10-d93bbd2064f5 None admin] RBAC: Authorizing `identity:create_grant(project_id=417e3e2f1333481795a089bef2e24bea, user_id=1eb1525233ba48c99b15d4cb75db4822, role_id=70c34ea493484322911dab62b4a56e75)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:33.416360 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f17aad7-f9cb-429a-8a10-d93bbd2064f5 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:33.417684 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ab01278-02b3-436f-abea-6a01c65ee05c None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 21:59:33.427271 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application [None req-1ab01278-02b3-436f-abea-6a01c65ee05c None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 13 21:59:33.427271 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 21:59:33.427271 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 21:59:33.427271 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 21:59:33.427271 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 21:59:33.427271 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 21:59:33.427271 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 21:59:33.427271 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 21:59:33.427271 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 21:59:33.427271 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 21:59:33.427271 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 21:59:33.427271 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 21:59:33.427271 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 21:59:33.427271 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 21:59:33.427271 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 21:59:33.427271 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 21:59:33.428442 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 21:59:33.428442 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 21:59:33.428442 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 21:59:33.428442 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 21:59:33.428442 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 21:59:33.428442 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 21:59:33.428442 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 21:59:33.428442 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 21:59:33.428442 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 21:59:33.428442 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 21:59:33.428442 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 21:59:33.428442 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application with Lock( Jul 13 21:59:33.428442 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 21:59:33.428442 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._enter() Jul 13 21:59:33.428442 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 21:59:33.428442 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 21:59:33.429616 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 21:59:33.429616 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.creator() Jul 13 21:59:33.429616 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 21:59:33.429616 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application created_value = creator( Jul 13 21:59:33.429616 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 21:59:33.429616 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 21:59:33.429616 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 21:59:33.429616 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 21:59:33.429616 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 21:59:33.429616 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 21:59:33.429616 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 13 21:59:33.429616 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Jul 13 21:59:33.432641 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 307/627] 10.222.0.52 () {56 vars in 1102 bytes} [Wed Jul 13 21:59:33 2022] GET /identity/v3/roles/service => generated 85 bytes in 229 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 21:59:33.438646 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 321/628] 10.222.0.52 () {58 vars in 1439 bytes} [Wed Jul 13 21:59:33 2022] PUT /identity/v3/projects/417e3e2f1333481795a089bef2e24bea/users/1eb1525233ba48c99b15d4cb75db4822/roles/70c34ea493484322911dab62b4a56e75 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 21:59:33.438843 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab01278-02b3-436f-abea-6a01c65ee05c None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:33.438926 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f17aad7-f9cb-429a-8a10-d93bbd2064f5 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:33.516143 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:59:33.573529 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:59:33.596070 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ab01278-02b3-436f-abea-6a01c65ee05c None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:33.597704 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f17aad7-f9cb-429a-8a10-d93bbd2064f5 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:33.638415 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c60abab-0bae-47be-867c-8ac5ff2ab09b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:33.639043 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c60abab-0bae-47be-867c-8ac5ff2ab09b None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:33.639324 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c60abab-0bae-47be-867c-8ac5ff2ab09b None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:33.639597 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c60abab-0bae-47be-867c-8ac5ff2ab09b None admin] PATH_INFO: `/v3/domains` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:33.640336 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c60abab-0bae-47be-867c-8ac5ff2ab09b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 21:59:33.640642 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c60abab-0bae-47be-867c-8ac5ff2ab09b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:33.641171 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381c2a4f-03ee-4ac7-8bac-43f3f1463fb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:33.641855 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381c2a4f-03ee-4ac7-8bac-43f3f1463fb2 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:33.642055 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c60abab-0bae-47be-867c-8ac5ff2ab09b None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:33.642153 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381c2a4f-03ee-4ac7-8bac-43f3f1463fb2 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:33.642401 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-381c2a4f-03ee-4ac7-8bac-43f3f1463fb2 None admin] PATH_INFO: `/v3/roles` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:33.643178 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381c2a4f-03ee-4ac7-8bac-43f3f1463fb2 None admin] RBAC: Adding query filter params (name=service) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 21:59:33.643749 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381c2a4f-03ee-4ac7-8bac-43f3f1463fb2 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:33.645557 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-381c2a4f-03ee-4ac7-8bac-43f3f1463fb2 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:33.655513 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 308/629] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Jul 13 21:59:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 222 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 21:59:33.656463 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 322/630] 10.222.0.52 () {56 vars in 1103 bytes} [Wed Jul 13 21:59:33 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 21:59:33.657389 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2bd381-e5d0-422d-b5d7-d0c9da23c5b6 None None] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:33.657866 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2bd381-e5d0-422d-b5d7-d0c9da23c5b6 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:33.658326 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a2bd381-e5d0-422d-b5d7-d0c9da23c5b6 None None] PATH_INFO: `/` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:33.660117 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 309/631] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jul 13 21:59:33 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 21:59:33.661163 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381c2a4f-03ee-4ac7-8bac-43f3f1463fb2 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:33.662092 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2bd381-e5d0-422d-b5d7-d0c9da23c5b6 None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:33.766273 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:59:33.773107 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a2bd381-e5d0-422d-b5d7-d0c9da23c5b6 None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:33.791185 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-381c2a4f-03ee-4ac7-8bac-43f3f1463fb2 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:33.810942 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-817b2da0-cc59-480a-8838-38c9bda37d93 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:33.812337 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817b2da0-cc59-480a-8838-38c9bda37d93 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:33.812807 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817b2da0-cc59-480a-8838-38c9bda37d93 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:33.813274 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-817b2da0-cc59-480a-8838-38c9bda37d93 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:33.827130 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-817b2da0-cc59-480a-8838-38c9bda37d93 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:33.828952 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-817b2da0-cc59-480a-8838-38c9bda37d93 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:33.831552 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798018ef-b88f-4a8e-aa6a-34cd50f524b0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:33.832679 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798018ef-b88f-4a8e-aa6a-34cd50f524b0 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:33.832965 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798018ef-b88f-4a8e-aa6a-34cd50f524b0 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:33.833240 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-798018ef-b88f-4a8e-aa6a-34cd50f524b0 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:33.842921 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798018ef-b88f-4a8e-aa6a-34cd50f524b0 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:33.844392 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-798018ef-b88f-4a8e-aa6a-34cd50f524b0 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:33.849454 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application [None req-817b2da0-cc59-480a-8838-38c9bda37d93 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 21:59:33.849454 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 21:59:33.849454 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 21:59:33.849454 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 21:59:33.849454 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 21:59:33.849454 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 21:59:33.849454 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 21:59:33.849454 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 21:59:33.849454 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 21:59:33.849454 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 21:59:33.849454 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 21:59:33.849454 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 21:59:33.849454 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 21:59:33.849454 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 21:59:33.849454 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 21:59:33.850322 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 21:59:33.850322 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 21:59:33.850322 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 21:59:33.850322 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 21:59:33.850322 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 21:59:33.850322 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 21:59:33.850322 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 21:59:33.850322 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 21:59:33.850322 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application with Lock( Jul 13 21:59:33.850322 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 21:59:33.850322 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._enter() Jul 13 21:59:33.850322 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 21:59:33.850322 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 21:59:33.850322 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 21:59:33.850322 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.creator() Jul 13 21:59:33.850322 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 21:59:33.850322 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application created_value = creator( Jul 13 21:59:33.851254 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 21:59:33.851254 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 21:59:33.851254 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 21:59:33.851254 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Jul 13 21:59:33.853538 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 310/632] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Jul 13 21:59:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 21:59:33.855280 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e019d459-432a-4d76-ba08-68b86eb10bb8 None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:33.855779 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e019d459-432a-4d76-ba08-68b86eb10bb8 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:33.856241 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e019d459-432a-4d76-ba08-68b86eb10bb8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:33.859314 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application [None req-798018ef-b88f-4a8e-aa6a-34cd50f524b0 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 13 21:59:33.859314 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 21:59:33.859314 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 21:59:33.859314 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 21:59:33.859314 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 21:59:33.859314 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 21:59:33.859314 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 21:59:33.859314 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 21:59:33.859314 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 21:59:33.859314 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 21:59:33.859314 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 21:59:33.859314 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 21:59:33.859314 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 13 21:59:33.859314 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 13 21:59:33.859314 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 13 21:59:33.859314 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 13 21:59:33.860248 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 21:59:33.860248 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 21:59:33.860248 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 13 21:59:33.860248 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 21:59:33.860248 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 13 21:59:33.860248 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 21:59:33.860248 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 21:59:33.860248 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 21:59:33.860248 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 21:59:33.860248 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 21:59:33.860248 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 21:59:33.860248 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application with Lock( Jul 13 21:59:33.860248 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 21:59:33.860248 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._enter() Jul 13 21:59:33.860248 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 21:59:33.860248 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 21:59:33.860248 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 21:59:33.860248 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.creator() Jul 13 21:59:33.861177 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 21:59:33.861177 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application created_value = creator( Jul 13 21:59:33.861177 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 13 21:59:33.861177 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 13 21:59:33.861177 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 13 21:59:33.861177 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 13 21:59:33.861177 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 13 21:59:33.861177 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 13 21:59:33.861177 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 13 21:59:33.861177 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Jul 13 21:59:33.861177 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 323/633] 10.222.0.52 () {56 vars in 1093 bytes} [Wed Jul 13 21:59:33 2022] GET /identity/v3/users/nova => generated 82 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 13 21:59:33.866341 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798018ef-b88f-4a8e-aa6a-34cd50f524b0 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:33.871250 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:59:33.881633 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-e019d459-432a-4d76-ba08-68b86eb10bb8 None None] MFA Rules not processed for user `cc99c93da5074cf5a48bc2aa9a92c5e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:59:33.883124 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-e019d459-432a-4d76-ba08-68b86eb10bb8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:59:33.957080 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 311/634] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jul 13 21:59:33 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 13 21:59:33.958473 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e019d459-432a-4d76-ba08-68b86eb10bb8 None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:33.988600 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-798018ef-b88f-4a8e-aa6a-34cd50f524b0 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:34.025158 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67442a47-f58e-4f9a-b4b8-2e66f3bca3be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:34.026300 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67442a47-f58e-4f9a-b4b8-2e66f3bca3be None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:34.026819 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67442a47-f58e-4f9a-b4b8-2e66f3bca3be None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:34.027387 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67442a47-f58e-4f9a-b4b8-2e66f3bca3be None admin] PATH_INFO: `/v3/domains` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:34.028404 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67442a47-f58e-4f9a-b4b8-2e66f3bca3be None admin] RBAC: Adding query filter params (name=Default) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 21:59:34.028917 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67442a47-f58e-4f9a-b4b8-2e66f3bca3be None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:34.030686 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67442a47-f58e-4f9a-b4b8-2e66f3bca3be None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:34.044552 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 324/635] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Jul 13 21:59:33 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 21:59:34.045799 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67442a47-f58e-4f9a-b4b8-2e66f3bca3be None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:34.056230 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:59:34.081471 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e019d459-432a-4d76-ba08-68b86eb10bb8 None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:34.113145 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:59:34.123610 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f71646d-e642-4d25-bf3e-37da23f406ef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:34.125086 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f71646d-e642-4d25-bf3e-37da23f406ef None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:34.125366 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f71646d-e642-4d25-bf3e-37da23f406ef None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:34.125643 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f71646d-e642-4d25-bf3e-37da23f406ef None admin] PATH_INFO: `/v3/users` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:34.126520 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f71646d-e642-4d25-bf3e-37da23f406ef None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 21:59:34.126833 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f71646d-e642-4d25-bf3e-37da23f406ef None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:34.128278 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f71646d-e642-4d25-bf3e-37da23f406ef None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:34.155280 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 312/636] 10.222.0.52 () {56 vars in 1133 bytes} [Wed Jul 13 21:59:33 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 21:59:34.155280 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f71646d-e642-4d25-bf3e-37da23f406ef None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:34.161776 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67442a47-f58e-4f9a-b4b8-2e66f3bca3be None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:34.198147 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ee0c8c-470f-49ac-8cb3-649aba52a4c5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:34.198983 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ee0c8c-470f-49ac-8cb3-649aba52a4c5 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:34.199606 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ee0c8c-470f-49ac-8cb3-649aba52a4c5 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:34.199873 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ee0c8c-470f-49ac-8cb3-649aba52a4c5 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:34.210596 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ee0c8c-470f-49ac-8cb3-649aba52a4c5 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 21:59:34.218339 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application [None req-d0ee0c8c-470f-49ac-8cb3-649aba52a4c5 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 13 21:59:34.218339 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 21:59:34.218339 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 21:59:34.218339 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 21:59:34.218339 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 21:59:34.218339 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 21:59:34.218339 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 21:59:34.218339 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 21:59:34.218339 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 21:59:34.218339 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 21:59:34.218339 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 21:59:34.218339 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 21:59:34.218339 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 21:59:34.218339 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 21:59:34.218339 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 21:59:34.218339 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 21:59:34.219309 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 21:59:34.219309 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 21:59:34.219309 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 21:59:34.219309 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 21:59:34.219309 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 21:59:34.219309 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 21:59:34.219309 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 21:59:34.219309 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 21:59:34.219309 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 21:59:34.219309 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 21:59:34.219309 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 21:59:34.219309 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application with Lock( Jul 13 21:59:34.219309 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 21:59:34.219309 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._enter() Jul 13 21:59:34.219309 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 21:59:34.219309 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 21:59:34.220156 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 21:59:34.220156 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.creator() Jul 13 21:59:34.220156 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 21:59:34.220156 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application created_value = creator( Jul 13 21:59:34.220156 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 21:59:34.220156 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 21:59:34.220156 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 21:59:34.220156 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 21:59:34.220156 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 21:59:34.220156 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 21:59:34.220156 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 13 21:59:34.220156 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Jul 13 21:59:34.220156 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 325/637] 10.222.0.52 () {56 vars in 1102 bytes} [Wed Jul 13 21:59:34 2022] GET /identity/v3/roles/service => generated 85 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 21:59:34.220846 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ee0c8c-470f-49ac-8cb3-649aba52a4c5 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:34.270513 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f71646d-e642-4d25-bf3e-37da23f406ef None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:34.304289 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8df074a-4367-41bf-bcb4-7d0e14ffa122 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:34.305074 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8df074a-4367-41bf-bcb4-7d0e14ffa122 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:34.305346 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8df074a-4367-41bf-bcb4-7d0e14ffa122 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:34.305614 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8df074a-4367-41bf-bcb4-7d0e14ffa122 None admin] PATH_INFO: `/v3/users/glance` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:34.312793 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8df074a-4367-41bf-bcb4-7d0e14ffa122 None admin] RBAC: Authorizing `identity:get_user(user_id=glance)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:34.314261 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8df074a-4367-41bf-bcb4-7d0e14ffa122 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:34.316135 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:59:34.331020 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application [None req-d8df074a-4367-41bf-bcb4-7d0e14ffa122 None admin] Could not find user: glance.: keystone.exception.UserNotFound: Could not find user: glance. Jul 13 21:59:34.331020 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 21:59:34.331020 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 21:59:34.331020 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 21:59:34.331020 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 21:59:34.331020 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 21:59:34.331020 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 21:59:34.331020 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 21:59:34.331020 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 21:59:34.331020 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 21:59:34.331020 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 21:59:34.331020 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 21:59:34.331020 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 13 21:59:34.331020 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 13 21:59:34.331020 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 13 21:59:34.331020 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 13 21:59:34.332036 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 21:59:34.332036 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 21:59:34.332036 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 13 21:59:34.332036 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 21:59:34.332036 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 13 21:59:34.332036 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 21:59:34.332036 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 21:59:34.332036 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 21:59:34.332036 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 21:59:34.332036 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 21:59:34.332036 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 21:59:34.332036 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application with Lock( Jul 13 21:59:34.332036 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 21:59:34.332036 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._enter() Jul 13 21:59:34.332036 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 21:59:34.332036 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 21:59:34.332036 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 21:59:34.332036 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.creator() Jul 13 21:59:34.332966 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 21:59:34.332966 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application created_value = creator( Jul 13 21:59:34.332966 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 13 21:59:34.332966 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 13 21:59:34.332966 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 13 21:59:34.332966 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 13 21:59:34.332966 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 13 21:59:34.332966 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 13 21:59:34.332966 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance. Jul 13 21:59:34.332966 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Jul 13 21:59:34.332966 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 313/638] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Jul 13 21:59:34 2022] GET /identity/v3/users/glance => generated 84 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 21:59:34.334286 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e3f1f8-ac84-4b79-831f-da48779e2592 None None] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:34.334286 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e3f1f8-ac84-4b79-831f-da48779e2592 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:34.334463 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2e3f1f8-ac84-4b79-831f-da48779e2592 None None] PATH_INFO: `/` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:34.335936 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 314/639] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jul 13 21:59:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 21:59:34.336460 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e3f1f8-ac84-4b79-831f-da48779e2592 None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:34.343806 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ee0c8c-470f-49ac-8cb3-649aba52a4c5 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:34.366875 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:59:34.379418 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b195395-6a43-4caf-8549-e1b7f685b421 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:34.380084 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b195395-6a43-4caf-8549-e1b7f685b421 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:34.380372 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b195395-6a43-4caf-8549-e1b7f685b421 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:34.380639 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b195395-6a43-4caf-8549-e1b7f685b421 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:34.445462 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2e3f1f8-ac84-4b79-831f-da48779e2592 None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:34.450866 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b195395-6a43-4caf-8549-e1b7f685b421 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:34.452357 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b195395-6a43-4caf-8549-e1b7f685b421 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:34.485020 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62f877c-58ca-4bfe-9e88-cbc4d000ed4d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:34.485609 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62f877c-58ca-4bfe-9e88-cbc4d000ed4d None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:34.485911 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62f877c-58ca-4bfe-9e88-cbc4d000ed4d None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:34.486144 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a62f877c-58ca-4bfe-9e88-cbc4d000ed4d None admin] PATH_INFO: `/v3/roles` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:34.487232 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62f877c-58ca-4bfe-9e88-cbc4d000ed4d None admin] RBAC: Adding query filter params (name=service) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 21:59:34.487232 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62f877c-58ca-4bfe-9e88-cbc4d000ed4d None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:34.488549 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a62f877c-58ca-4bfe-9e88-cbc4d000ed4d None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:34.497179 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 315/640] 10.222.0.52 () {56 vars in 1103 bytes} [Wed Jul 13 21:59:34 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 21:59:34.497877 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62f877c-58ca-4bfe-9e88-cbc4d000ed4d None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:34.558075 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:59:34.568158 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 326/641] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Jul 13 21:59:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 348 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 13 21:59:34.569619 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a08ad81-8d3e-4859-a7dd-3ceac74d457f None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:34.569965 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a08ad81-8d3e-4859-a7dd-3ceac74d457f None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:34.570302 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a08ad81-8d3e-4859-a7dd-3ceac74d457f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:34.588075 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-9a08ad81-8d3e-4859-a7dd-3ceac74d457f None None] MFA Rules not processed for user `cc99c93da5074cf5a48bc2aa9a92c5e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:59:34.589363 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-9a08ad81-8d3e-4859-a7dd-3ceac74d457f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:59:34.606482 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a62f877c-58ca-4bfe-9e88-cbc4d000ed4d None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:34.608539 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:59:34.643455 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9ad169-b28e-4c9d-99bd-d9330d78ceb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:34.644130 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9ad169-b28e-4c9d-99bd-d9330d78ceb4 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:34.644406 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9ad169-b28e-4c9d-99bd-d9330d78ceb4 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:34.644673 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff9ad169-b28e-4c9d-99bd-d9330d78ceb4 None admin] PATH_INFO: `/v3/users` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:34.645512 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9ad169-b28e-4c9d-99bd-d9330d78ceb4 None admin] RBAC: Adding query filter params (domain_id=default, name=glance) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 21:59:34.645813 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9ad169-b28e-4c9d-99bd-d9330d78ceb4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:34.647266 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff9ad169-b28e-4c9d-99bd-d9330d78ceb4 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:34.664352 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 327/642] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jul 13 21:59:34 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 13 21:59:34.665989 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a08ad81-8d3e-4859-a7dd-3ceac74d457f None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:34.669290 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 316/643] 10.222.0.52 () {56 vars in 1137 bytes} [Wed Jul 13 21:59:34 2022] GET /identity/v3/users?domain_id=default&name=glance => generated 372 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 21:59:34.670194 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9ad169-b28e-4c9d-99bd-d9330d78ceb4 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:34.770028 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff9ad169-b28e-4c9d-99bd-d9330d78ceb4 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:34.778577 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a08ad81-8d3e-4859-a7dd-3ceac74d457f None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:34.789078 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:59:34.807573 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e28c3cf-7d03-4ffd-82de-c1141566e215 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:34.808385 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e28c3cf-7d03-4ffd-82de-c1141566e215 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:34.808664 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e28c3cf-7d03-4ffd-82de-c1141566e215 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:34.808996 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e28c3cf-7d03-4ffd-82de-c1141566e215 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:34.816371 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9743294-7492-4daa-b075-494638cc810e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:34.817622 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9743294-7492-4daa-b075-494638cc810e None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:34.818070 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9743294-7492-4daa-b075-494638cc810e None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:34.818538 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9743294-7492-4daa-b075-494638cc810e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:34.827077 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e28c3cf-7d03-4ffd-82de-c1141566e215 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:34.829652 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e28c3cf-7d03-4ffd-82de-c1141566e215 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:34.835387 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9743294-7492-4daa-b075-494638cc810e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:34.838277 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9743294-7492-4daa-b075-494638cc810e None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:34.859582 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application [None req-2e28c3cf-7d03-4ffd-82de-c1141566e215 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 21:59:34.859582 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 21:59:34.859582 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 21:59:34.859582 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 21:59:34.859582 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 21:59:34.859582 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 21:59:34.859582 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 21:59:34.859582 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 21:59:34.859582 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 21:59:34.859582 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 21:59:34.859582 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 21:59:34.859582 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 21:59:34.859582 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 21:59:34.859582 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 21:59:34.859582 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 21:59:34.860564 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 21:59:34.860564 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 21:59:34.860564 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 21:59:34.860564 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 21:59:34.860564 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 21:59:34.860564 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 21:59:34.860564 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 21:59:34.860564 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 21:59:34.860564 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application with Lock( Jul 13 21:59:34.860564 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 21:59:34.860564 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._enter() Jul 13 21:59:34.860564 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 21:59:34.860564 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 21:59:34.860564 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 21:59:34.860564 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.creator() Jul 13 21:59:34.860564 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 21:59:34.860564 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application created_value = creator( Jul 13 21:59:34.861512 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 21:59:34.861512 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 21:59:34.861512 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 21:59:34.861512 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Jul 13 21:59:34.862222 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 317/644] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Jul 13 21:59:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 21:59:34.863143 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e28c3cf-7d03-4ffd-82de-c1141566e215 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:34.865845 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application [None req-e9743294-7492-4daa-b075-494638cc810e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 21:59:34.865845 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 21:59:34.865845 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 21:59:34.865845 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 21:59:34.865845 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 21:59:34.865845 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 21:59:34.865845 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 21:59:34.865845 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 21:59:34.865845 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 21:59:34.865845 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 21:59:34.865845 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 21:59:34.865845 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 21:59:34.865845 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 21:59:34.865845 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 21:59:34.865845 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 21:59:34.866823 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 21:59:34.866823 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 21:59:34.866823 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 21:59:34.866823 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 21:59:34.866823 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 21:59:34.866823 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 21:59:34.866823 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 21:59:34.866823 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 21:59:34.866823 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application with Lock( Jul 13 21:59:34.866823 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 21:59:34.866823 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._enter() Jul 13 21:59:34.866823 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 21:59:34.866823 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 21:59:34.866823 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 21:59:34.866823 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.creator() Jul 13 21:59:34.866823 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 21:59:34.866823 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application created_value = creator( Jul 13 21:59:34.868036 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 21:59:34.868036 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 21:59:34.868036 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 21:59:34.868036 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Jul 13 21:59:34.870705 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 328/645] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Jul 13 21:59:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 21:59:34.871379 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9743294-7492-4daa-b075-494638cc810e None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:34.880044 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:59:34.975532 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e28c3cf-7d03-4ffd-82de-c1141566e215 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:34.976954 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9743294-7492-4daa-b075-494638cc810e None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:35.011256 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b0e653-35b1-4ee7-8a02-7ff967581771 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:35.012183 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b0e653-35b1-4ee7-8a02-7ff967581771 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:35.012635 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b0e653-35b1-4ee7-8a02-7ff967581771 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:35.012895 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84b0e653-35b1-4ee7-8a02-7ff967581771 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:35.017430 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ce6796-d1cc-4538-a405-494b0bfd298e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:35.018236 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ce6796-d1cc-4538-a405-494b0bfd298e None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:35.018535 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ce6796-d1cc-4538-a405-494b0bfd298e None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:35.018810 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5ce6796-d1cc-4538-a405-494b0bfd298e None admin] PATH_INFO: `/v3/roles/service` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:35.030675 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5ce6796-d1cc-4538-a405-494b0bfd298e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 21:59:35.038289 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application [None req-b5ce6796-d1cc-4538-a405-494b0bfd298e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 13 21:59:35.038289 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 21:59:35.038289 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 21:59:35.038289 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 21:59:35.038289 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 21:59:35.038289 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 21:59:35.038289 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 21:59:35.038289 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 21:59:35.038289 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 21:59:35.038289 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 21:59:35.038289 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 21:59:35.038289 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 21:59:35.038289 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 21:59:35.038289 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 21:59:35.038289 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 21:59:35.038289 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 21:59:35.039286 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 21:59:35.039286 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 21:59:35.039286 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 21:59:35.039286 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 21:59:35.039286 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 21:59:35.039286 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 21:59:35.039286 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 21:59:35.039286 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 21:59:35.039286 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 21:59:35.039286 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 21:59:35.039286 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 21:59:35.039286 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application with Lock( Jul 13 21:59:35.039286 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 21:59:35.039286 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._enter() Jul 13 21:59:35.039286 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 21:59:35.039286 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 21:59:35.040181 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 21:59:35.040181 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.creator() Jul 13 21:59:35.040181 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 21:59:35.040181 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application created_value = creator( Jul 13 21:59:35.040181 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 21:59:35.040181 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 21:59:35.040181 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 21:59:35.040181 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 21:59:35.040181 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 21:59:35.040181 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 21:59:35.040181 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 13 21:59:35.040181 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Jul 13 21:59:35.040181 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 329/646] 10.222.0.52 () {56 vars in 1102 bytes} [Wed Jul 13 21:59:34 2022] GET /identity/v3/roles/service => generated 85 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 21:59:35.041085 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ce6796-d1cc-4538-a405-494b0bfd298e None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:35.061246 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:59:35.091773 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b0e653-35b1-4ee7-8a02-7ff967581771 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:35.093219 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-84b0e653-35b1-4ee7-8a02-7ff967581771 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:35.109707 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:59:35.148821 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5ce6796-d1cc-4538-a405-494b0bfd298e None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:35.185015 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef00ef7-7cca-46fb-aabf-3ce225e26996 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:35.186075 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef00ef7-7cca-46fb-aabf-3ce225e26996 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:35.186604 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef00ef7-7cca-46fb-aabf-3ce225e26996 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:35.186963 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eef00ef7-7cca-46fb-aabf-3ce225e26996 None admin] PATH_INFO: `/v3/domains` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:35.187739 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef00ef7-7cca-46fb-aabf-3ce225e26996 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 21:59:35.188108 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef00ef7-7cca-46fb-aabf-3ce225e26996 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:35.189700 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eef00ef7-7cca-46fb-aabf-3ce225e26996 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:35.196608 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 318/647] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Jul 13 21:59:34 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 13 21:59:35.200327 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b0e653-35b1-4ee7-8a02-7ff967581771 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:35.202810 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 330/648] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Jul 13 21:59:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 21:59:35.203793 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef00ef7-7cca-46fb-aabf-3ce225e26996 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:35.276246 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:59:35.307241 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eef00ef7-7cca-46fb-aabf-3ce225e26996 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:35.314170 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84b0e653-35b1-4ee7-8a02-7ff967581771 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:35.332423 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:59:35.341767 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1227baab-8f56-4a4e-941a-9061c71eb1d6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:35.342443 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1227baab-8f56-4a4e-941a-9061c71eb1d6 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:35.342742 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1227baab-8f56-4a4e-941a-9061c71eb1d6 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:35.343074 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1227baab-8f56-4a4e-941a-9061c71eb1d6 None admin] PATH_INFO: `/v3/roles` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:35.343827 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1227baab-8f56-4a4e-941a-9061c71eb1d6 None admin] RBAC: Adding query filter params (name=service) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 21:59:35.344175 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1227baab-8f56-4a4e-941a-9061c71eb1d6 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:35.345548 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1227baab-8f56-4a4e-941a-9061c71eb1d6 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:35.353851 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 331/649] 10.222.0.52 () {56 vars in 1103 bytes} [Wed Jul 13 21:59:35 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 21:59:35.354316 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a413bf-5f47-4104-ba24-883660d7424c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:35.354663 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1227baab-8f56-4a4e-941a-9061c71eb1d6 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:35.355382 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a413bf-5f47-4104-ba24-883660d7424c None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:35.355660 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a413bf-5f47-4104-ba24-883660d7424c None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:35.355923 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a413bf-5f47-4104-ba24-883660d7424c None admin] PATH_INFO: `/v3/domains` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:35.356787 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a413bf-5f47-4104-ba24-883660d7424c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 21:59:35.357092 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a413bf-5f47-4104-ba24-883660d7424c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:35.358474 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a413bf-5f47-4104-ba24-883660d7424c None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:35.370895 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 319/650] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Jul 13 21:59:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 21:59:35.371719 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a413bf-5f47-4104-ba24-883660d7424c None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:35.471422 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1227baab-8f56-4a4e-941a-9061c71eb1d6 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:35.478125 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a413bf-5f47-4104-ba24-883660d7424c None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:35.511656 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:59:35.511917 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d93c18d-a67f-4e34-83d4-7e0b5eed7cf9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:35.513114 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d93c18d-a67f-4e34-83d4-7e0b5eed7cf9 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:35.513239 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d93c18d-a67f-4e34-83d4-7e0b5eed7cf9 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:35.513487 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d93c18d-a67f-4e34-83d4-7e0b5eed7cf9 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:35.518939 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f0b4a8-5f41-4db1-8b60-e5d405a14655 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:35.519745 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f0b4a8-5f41-4db1-8b60-e5d405a14655 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:35.520024 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f0b4a8-5f41-4db1-8b60-e5d405a14655 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:35.520296 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7f0b4a8-5f41-4db1-8b60-e5d405a14655 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:35.525413 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d93c18d-a67f-4e34-83d4-7e0b5eed7cf9 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:35.526788 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d93c18d-a67f-4e34-83d4-7e0b5eed7cf9 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:35.527934 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f0b4a8-5f41-4db1-8b60-e5d405a14655 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:35.529386 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7f0b4a8-5f41-4db1-8b60-e5d405a14655 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:35.543765 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application [None req-2d93c18d-a67f-4e34-83d4-7e0b5eed7cf9 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 21:59:35.543765 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 21:59:35.543765 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 21:59:35.543765 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 21:59:35.543765 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 21:59:35.543765 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 21:59:35.543765 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 21:59:35.543765 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 21:59:35.543765 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 21:59:35.543765 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 21:59:35.543765 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 21:59:35.543765 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 21:59:35.543765 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 21:59:35.543765 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 21:59:35.543765 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 21:59:35.544681 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 21:59:35.544681 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 21:59:35.544681 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 21:59:35.544681 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 21:59:35.544681 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 21:59:35.544681 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 21:59:35.544681 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 21:59:35.544681 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 21:59:35.544681 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application with Lock( Jul 13 21:59:35.544681 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 21:59:35.544681 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._enter() Jul 13 21:59:35.544681 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 21:59:35.544681 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 21:59:35.544681 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 21:59:35.544681 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.creator() Jul 13 21:59:35.544681 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 21:59:35.544681 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application created_value = creator( Jul 13 21:59:35.545585 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 21:59:35.545585 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 21:59:35.545585 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 21:59:35.545585 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Jul 13 21:59:35.545585 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 332/651] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Jul 13 21:59:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 21:59:35.546121 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d93c18d-a67f-4e34-83d4-7e0b5eed7cf9 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:35.550113 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application [None req-d7f0b4a8-5f41-4db1-8b60-e5d405a14655 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 13 21:59:35.550113 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 21:59:35.550113 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 21:59:35.550113 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 21:59:35.550113 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 21:59:35.550113 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 21:59:35.550113 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 21:59:35.550113 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 21:59:35.550113 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 21:59:35.550113 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 21:59:35.550113 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 21:59:35.550113 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 21:59:35.550113 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 13 21:59:35.550113 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 13 21:59:35.550113 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 13 21:59:35.551117 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 13 21:59:35.551117 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 21:59:35.551117 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 21:59:35.551117 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 21:59:35.551117 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 21:59:35.551117 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 21:59:35.551117 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 21:59:35.551117 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 21:59:35.551117 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application with Lock( Jul 13 21:59:35.551117 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 21:59:35.551117 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._enter() Jul 13 21:59:35.551117 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 21:59:35.551117 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 21:59:35.551117 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 21:59:35.551117 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.creator() Jul 13 21:59:35.551117 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 21:59:35.551117 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application created_value = creator( Jul 13 21:59:35.551994 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 13 21:59:35.551994 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 13 21:59:35.551994 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 13 21:59:35.551994 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 13 21:59:35.551994 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 13 21:59:35.551994 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 13 21:59:35.551994 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 13 21:59:35.551994 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Jul 13 21:59:35.551994 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 320/652] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Jul 13 21:59:35 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 21:59:35.551994 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f0b4a8-5f41-4db1-8b60-e5d405a14655 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:35.593823 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:59:35.653544 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d93c18d-a67f-4e34-83d4-7e0b5eed7cf9 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:35.654621 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7f0b4a8-5f41-4db1-8b60-e5d405a14655 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:35.686507 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0564c17-837f-461b-a30c-edfdbc5465dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:35.687315 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0564c17-837f-461b-a30c-edfdbc5465dd None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:35.687594 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0564c17-837f-461b-a30c-edfdbc5465dd None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:35.687830 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87ddad7d-f54f-4be5-95cc-214c3b607659 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:35.687954 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0564c17-837f-461b-a30c-edfdbc5465dd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:35.688624 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ddad7d-f54f-4be5-95cc-214c3b607659 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:35.688898 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ddad7d-f54f-4be5-95cc-214c3b607659 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:35.689163 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87ddad7d-f54f-4be5-95cc-214c3b607659 None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:35.696583 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ddad7d-f54f-4be5-95cc-214c3b607659 None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:35.697926 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87ddad7d-f54f-4be5-95cc-214c3b607659 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:35.698264 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0564c17-837f-461b-a30c-edfdbc5465dd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:35.700070 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0564c17-837f-461b-a30c-edfdbc5465dd None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:35.712675 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application [None req-87ddad7d-f54f-4be5-95cc-214c3b607659 None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 13 21:59:35.712675 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 21:59:35.712675 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 21:59:35.712675 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 21:59:35.712675 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 21:59:35.712675 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 21:59:35.712675 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 21:59:35.712675 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 21:59:35.712675 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 21:59:35.712675 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 21:59:35.712675 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 21:59:35.712675 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 21:59:35.712675 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 13 21:59:35.712675 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 13 21:59:35.712675 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 13 21:59:35.712675 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 13 21:59:35.713736 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 21:59:35.713736 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 21:59:35.713736 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 13 21:59:35.713736 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 21:59:35.713736 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 13 21:59:35.713736 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 21:59:35.713736 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 21:59:35.713736 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 21:59:35.713736 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 21:59:35.713736 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 21:59:35.713736 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 21:59:35.713736 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application with Lock( Jul 13 21:59:35.713736 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 21:59:35.713736 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._enter() Jul 13 21:59:35.713736 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 21:59:35.713736 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 21:59:35.713736 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 21:59:35.713736 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.creator() Jul 13 21:59:35.714739 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 21:59:35.714739 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application created_value = creator( Jul 13 21:59:35.714739 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 13 21:59:35.714739 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 13 21:59:35.714739 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 13 21:59:35.714739 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 13 21:59:35.714739 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 13 21:59:35.714739 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 13 21:59:35.714739 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 13 21:59:35.714739 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Jul 13 21:59:35.714739 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 321/653] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Jul 13 21:59:35 2022] GET /identity/v3/users/cinder => generated 84 bytes in 163 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 21:59:35.715714 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c59e76-bffe-43d8-b25f-ec59232e9c9a None None] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:35.715714 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c59e76-bffe-43d8-b25f-ec59232e9c9a None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:35.715714 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60c59e76-bffe-43d8-b25f-ec59232e9c9a None None] PATH_INFO: `/` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:35.719149 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 322/654] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jul 13 21:59:35 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 21:59:35.719305 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application [None req-c0564c17-837f-461b-a30c-edfdbc5465dd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 21:59:35.719305 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 21:59:35.719305 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 21:59:35.719305 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 21:59:35.719305 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 21:59:35.719305 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 21:59:35.719305 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 21:59:35.719305 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 21:59:35.719305 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 21:59:35.719305 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 21:59:35.719305 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 21:59:35.719305 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 21:59:35.719305 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 21:59:35.719305 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 21:59:35.719305 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 21:59:35.720121 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 21:59:35.720121 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 21:59:35.720121 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 21:59:35.720121 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 21:59:35.720121 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 21:59:35.720121 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 21:59:35.720121 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 21:59:35.720121 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 21:59:35.720121 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application with Lock( Jul 13 21:59:35.720121 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 21:59:35.720121 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._enter() Jul 13 21:59:35.720121 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 21:59:35.720121 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 21:59:35.720121 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 21:59:35.720121 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.creator() Jul 13 21:59:35.720121 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 21:59:35.720121 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application created_value = creator( Jul 13 21:59:35.721014 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 21:59:35.721014 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 21:59:35.721014 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 21:59:35.721014 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Jul 13 21:59:35.721234 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c59e76-bffe-43d8-b25f-ec59232e9c9a None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:35.721310 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 333/655] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Jul 13 21:59:35 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 21:59:35.721310 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0564c17-837f-461b-a30c-edfdbc5465dd None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:35.773418 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:59:35.814468 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:59:35.823509 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60c59e76-bffe-43d8-b25f-ec59232e9c9a None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:35.824360 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0564c17-837f-461b-a30c-edfdbc5465dd None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:35.856616 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc2eb25-dd51-4c6c-8ace-bab106a2b8c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:35.857204 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc2eb25-dd51-4c6c-8ace-bab106a2b8c9 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:35.857325 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003ace7b-a6dc-4e63-8700-3cbb2f671a6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:35.857482 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc2eb25-dd51-4c6c-8ace-bab106a2b8c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:35.857750 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bc2eb25-dd51-4c6c-8ace-bab106a2b8c9 None admin] PATH_INFO: `/v3/domains` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:35.857874 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003ace7b-a6dc-4e63-8700-3cbb2f671a6c None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:35.858159 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003ace7b-a6dc-4e63-8700-3cbb2f671a6c None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:35.858437 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003ace7b-a6dc-4e63-8700-3cbb2f671a6c None admin] PATH_INFO: `/v3/projects` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:35.858553 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc2eb25-dd51-4c6c-8ace-bab106a2b8c9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 21:59:35.858792 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc2eb25-dd51-4c6c-8ace-bab106a2b8c9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:35.859259 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-003ace7b-a6dc-4e63-8700-3cbb2f671a6c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 21:59:35.859571 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-003ace7b-a6dc-4e63-8700-3cbb2f671a6c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:35.860164 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bc2eb25-dd51-4c6c-8ace-bab106a2b8c9 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:35.861009 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-003ace7b-a6dc-4e63-8700-3cbb2f671a6c None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:35.872205 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 323/656] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Jul 13 21:59:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 21:59:35.872987 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc2eb25-dd51-4c6c-8ace-bab106a2b8c9 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:35.873593 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 334/657] 10.222.0.52 () {56 vars in 1148 bytes} [Wed Jul 13 21:59:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 21:59:35.874845 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2520cc5-6eab-4cf5-87f9-ffd9a1cbcd89 None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:35.875136 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2520cc5-6eab-4cf5-87f9-ffd9a1cbcd89 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:35.875427 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2520cc5-6eab-4cf5-87f9-ffd9a1cbcd89 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:35.896434 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-d2520cc5-6eab-4cf5-87f9-ffd9a1cbcd89 None None] MFA Rules not processed for user `cc99c93da5074cf5a48bc2aa9a92c5e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:59:35.897628 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-d2520cc5-6eab-4cf5-87f9-ffd9a1cbcd89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:59:35.966313 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 335/658] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jul 13 21:59:35 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 13 21:59:35.967391 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2520cc5-6eab-4cf5-87f9-ffd9a1cbcd89 None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:35.971714 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bc2eb25-dd51-4c6c-8ace-bab106a2b8c9 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:35.982509 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:59:36.009344 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f55a202-51f8-46da-9bd4-715a035b3c59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:36.010024 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f55a202-51f8-46da-9bd4-715a035b3c59 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:36.011139 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f55a202-51f8-46da-9bd4-715a035b3c59 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:36.011139 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f55a202-51f8-46da-9bd4-715a035b3c59 None admin] PATH_INFO: `/v3/users` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:36.011437 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f55a202-51f8-46da-9bd4-715a035b3c59 None admin] RBAC: Adding query filter params (domain_id=default, name=cinder) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 21:59:36.011727 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f55a202-51f8-46da-9bd4-715a035b3c59 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:36.013162 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f55a202-51f8-46da-9bd4-715a035b3c59 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:36.030335 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 324/659] 10.222.0.52 () {56 vars in 1137 bytes} [Wed Jul 13 21:59:35 2022] GET /identity/v3/users?domain_id=default&name=cinder => generated 372 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 21:59:36.030994 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f55a202-51f8-46da-9bd4-715a035b3c59 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:36.058674 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:59:36.078561 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2520cc5-6eab-4cf5-87f9-ffd9a1cbcd89 None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:36.113668 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf99c89-781d-4f37-a9d4-1e56fb2cef89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:36.114371 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf99c89-781d-4f37-a9d4-1e56fb2cef89 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:36.114682 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf99c89-781d-4f37-a9d4-1e56fb2cef89 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:36.114950 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caf99c89-781d-4f37-a9d4-1e56fb2cef89 None admin] PATH_INFO: `/v3/domains` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:36.115671 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf99c89-781d-4f37-a9d4-1e56fb2cef89 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 21:59:36.115969 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf99c89-781d-4f37-a9d4-1e56fb2cef89 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:36.117339 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caf99c89-781d-4f37-a9d4-1e56fb2cef89 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:36.128635 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 336/660] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Jul 13 21:59:35 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 21:59:36.129417 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf99c89-781d-4f37-a9d4-1e56fb2cef89 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:36.134777 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f55a202-51f8-46da-9bd4-715a035b3c59 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:36.167847 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a3d9a1-7086-4e1d-a274-63fbda4c3c26 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:36.168496 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a3d9a1-7086-4e1d-a274-63fbda4c3c26 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:36.168773 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a3d9a1-7086-4e1d-a274-63fbda4c3c26 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:36.169046 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81a3d9a1-7086-4e1d-a274-63fbda4c3c26 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:36.169873 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a3d9a1-7086-4e1d-a274-63fbda4c3c26 None admin] RBAC: Adding query filter params (role.id=70c34ea493484322911dab62b4a56e75, scope.project.id=417e3e2f1333481795a089bef2e24bea, user.id=f84281091687464990ffbca590b9343b) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 21:59:36.170186 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a3d9a1-7086-4e1d-a274-63fbda4c3c26 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:36.171654 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81a3d9a1-7086-4e1d-a274-63fbda4c3c26 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:36.177458 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 325/661] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Jul 13 21:59:36 2022] GET /identity/v3/role_assignments?user.id=f84281091687464990ffbca590b9343b&scope.project.id=417e3e2f1333481795a089bef2e24bea&role.id=70c34ea493484322911dab62b4a56e75 => generated 608 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 21:59:36.178218 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a3d9a1-7086-4e1d-a274-63fbda4c3c26 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:36.217494 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:59:36.246303 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caf99c89-781d-4f37-a9d4-1e56fb2cef89 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:36.287434 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3eab79-6d60-45fe-b6f6-b2f958d75777 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:36.288236 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3eab79-6d60-45fe-b6f6-b2f958d75777 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:36.288531 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3eab79-6d60-45fe-b6f6-b2f958d75777 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:36.288804 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f3eab79-6d60-45fe-b6f6-b2f958d75777 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:36.292811 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81a3d9a1-7086-4e1d-a274-63fbda4c3c26 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:36.297351 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f3eab79-6d60-45fe-b6f6-b2f958d75777 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:36.298873 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f3eab79-6d60-45fe-b6f6-b2f958d75777 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:36.315145 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application [None req-9f3eab79-6d60-45fe-b6f6-b2f958d75777 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 13 21:59:36.315145 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 21:59:36.315145 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 21:59:36.315145 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 21:59:36.315145 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 21:59:36.315145 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 21:59:36.315145 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 21:59:36.315145 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 21:59:36.315145 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 21:59:36.315145 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 21:59:36.315145 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 21:59:36.315145 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 21:59:36.315145 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 13 21:59:36.315145 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 13 21:59:36.315145 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 13 21:59:36.316325 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 13 21:59:36.316325 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 21:59:36.316325 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 21:59:36.316325 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 21:59:36.316325 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 21:59:36.316325 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 21:59:36.316325 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 21:59:36.316325 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 21:59:36.316325 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application with Lock( Jul 13 21:59:36.316325 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 21:59:36.316325 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._enter() Jul 13 21:59:36.316325 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 21:59:36.316325 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 21:59:36.316325 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 21:59:36.316325 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.creator() Jul 13 21:59:36.316325 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 21:59:36.316325 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application created_value = creator( Jul 13 21:59:36.317203 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 13 21:59:36.317203 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 13 21:59:36.317203 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 13 21:59:36.317203 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 13 21:59:36.317203 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 13 21:59:36.317203 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 13 21:59:36.317203 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 13 21:59:36.317203 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Jul 13 21:59:36.317203 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 337/662] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Jul 13 21:59:36 2022] GET /identity/v3/projects/service => generated 88 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 21:59:36.317203 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3eab79-6d60-45fe-b6f6-b2f958d75777 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:36.321085 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:59:36.338733 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ca6834-d723-403b-8138-b522c6688331 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:36.339552 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ca6834-d723-403b-8138-b522c6688331 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:36.339835 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ca6834-d723-403b-8138-b522c6688331 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:36.340109 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40ca6834-d723-403b-8138-b522c6688331 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:36.353144 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40ca6834-d723-403b-8138-b522c6688331 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 21:59:36.360865 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application [None req-40ca6834-d723-403b-8138-b522c6688331 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 13 21:59:36.360865 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 21:59:36.360865 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 21:59:36.360865 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 21:59:36.360865 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 21:59:36.360865 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 21:59:36.360865 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 21:59:36.360865 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 21:59:36.360865 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 21:59:36.360865 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 21:59:36.360865 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 21:59:36.360865 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 21:59:36.360865 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 21:59:36.360865 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 21:59:36.360865 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 21:59:36.360865 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 21:59:36.362019 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 21:59:36.362019 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 21:59:36.362019 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 21:59:36.362019 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 21:59:36.362019 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 21:59:36.362019 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 21:59:36.362019 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 21:59:36.362019 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 21:59:36.362019 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 21:59:36.362019 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 21:59:36.362019 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 21:59:36.362019 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application with Lock( Jul 13 21:59:36.362019 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 21:59:36.362019 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._enter() Jul 13 21:59:36.362019 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 21:59:36.362019 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 21:59:36.362950 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 21:59:36.362950 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.creator() Jul 13 21:59:36.362950 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 21:59:36.362950 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application created_value = creator( Jul 13 21:59:36.362950 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 21:59:36.362950 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 21:59:36.362950 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 21:59:36.362950 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 21:59:36.362950 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 21:59:36.362950 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 21:59:36.362950 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 13 21:59:36.362950 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Jul 13 21:59:36.362950 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 326/663] 10.222.0.52 () {56 vars in 1102 bytes} [Wed Jul 13 21:59:36 2022] GET /identity/v3/roles/service => generated 85 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 21:59:36.363763 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ca6834-d723-403b-8138-b522c6688331 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:36.433929 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f3eab79-6d60-45fe-b6f6-b2f958d75777 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:36.466565 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:59:36.472890 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0266f82a-0362-4fd2-9dc6-af18cb944573 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:36.473557 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0266f82a-0362-4fd2-9dc6-af18cb944573 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:36.473832 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0266f82a-0362-4fd2-9dc6-af18cb944573 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:36.474314 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0266f82a-0362-4fd2-9dc6-af18cb944573 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:36.477157 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40ca6834-d723-403b-8138-b522c6688331 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:36.511593 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d649d7-c3d8-4b43-ba89-e76ab4741f91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:36.512398 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d649d7-c3d8-4b43-ba89-e76ab4741f91 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:36.512678 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d649d7-c3d8-4b43-ba89-e76ab4741f91 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:36.512950 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d649d7-c3d8-4b43-ba89-e76ab4741f91 None admin] PATH_INFO: `/v3/users/neutron` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:36.521279 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d649d7-c3d8-4b43-ba89-e76ab4741f91 None admin] RBAC: Authorizing `identity:get_user(user_id=neutron)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:36.522813 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d649d7-c3d8-4b43-ba89-e76ab4741f91 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:36.537224 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application [None req-c3d649d7-c3d8-4b43-ba89-e76ab4741f91 None admin] Could not find user: neutron.: keystone.exception.UserNotFound: Could not find user: neutron. Jul 13 21:59:36.537224 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 21:59:36.537224 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 21:59:36.537224 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 21:59:36.537224 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 21:59:36.537224 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 21:59:36.537224 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 21:59:36.537224 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 21:59:36.537224 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 21:59:36.537224 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 21:59:36.537224 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 21:59:36.537224 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 21:59:36.537224 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 13 21:59:36.537224 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 13 21:59:36.537224 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 13 21:59:36.537224 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 13 21:59:36.538215 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 21:59:36.538215 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 21:59:36.538215 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 13 21:59:36.538215 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 21:59:36.538215 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 13 21:59:36.538215 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 21:59:36.538215 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 21:59:36.538215 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 21:59:36.538215 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 21:59:36.538215 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 21:59:36.538215 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 21:59:36.538215 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application with Lock( Jul 13 21:59:36.538215 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 21:59:36.538215 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._enter() Jul 13 21:59:36.538215 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 21:59:36.538215 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 21:59:36.538215 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 21:59:36.538215 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.creator() Jul 13 21:59:36.539191 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 21:59:36.539191 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application created_value = creator( Jul 13 21:59:36.539191 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 13 21:59:36.539191 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 13 21:59:36.539191 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 13 21:59:36.539191 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 13 21:59:36.539191 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 13 21:59:36.539191 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 13 21:59:36.539191 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: neutron. Jul 13 21:59:36.539191 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Jul 13 21:59:36.543301 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 327/664] 10.222.0.52 () {56 vars in 1102 bytes} [Wed Jul 13 21:59:36 2022] GET /identity/v3/users/neutron => generated 85 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 21:59:36.544204 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d649d7-c3d8-4b43-ba89-e76ab4741f91 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:36.549893 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0266f82a-0362-4fd2-9dc6-af18cb944573 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:36.549893 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0266f82a-0362-4fd2-9dc6-af18cb944573 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:36.557759 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:59:36.651791 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d649d7-c3d8-4b43-ba89-e76ab4741f91 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:36.656129 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 338/665] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Jul 13 21:59:36 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 13 21:59:36.656987 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0266f82a-0362-4fd2-9dc6-af18cb944573 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:36.692979 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638574a2-2143-4ab0-997d-c9b465a9a155 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:36.694337 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638574a2-2143-4ab0-997d-c9b465a9a155 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:36.694337 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638574a2-2143-4ab0-997d-c9b465a9a155 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:36.694337 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-638574a2-2143-4ab0-997d-c9b465a9a155 None admin] PATH_INFO: `/v3/projects` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:36.694973 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638574a2-2143-4ab0-997d-c9b465a9a155 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 21:59:36.695277 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638574a2-2143-4ab0-997d-c9b465a9a155 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:36.696914 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-638574a2-2143-4ab0-997d-c9b465a9a155 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:36.715102 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 328/666] 10.222.0.52 () {56 vars in 1148 bytes} [Wed Jul 13 21:59:36 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 21:59:36.715102 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638574a2-2143-4ab0-997d-c9b465a9a155 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:36.730985 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:59:36.769907 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0266f82a-0362-4fd2-9dc6-af18cb944573 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:36.771627 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:59:36.809612 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21492c1d-ac95-4ea8-ba41-67eb1d6956ec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:36.810286 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21492c1d-ac95-4ea8-ba41-67eb1d6956ec None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:36.810597 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21492c1d-ac95-4ea8-ba41-67eb1d6956ec None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:36.810870 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21492c1d-ac95-4ea8-ba41-67eb1d6956ec None admin] PATH_INFO: `/v3/roles` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:36.811734 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21492c1d-ac95-4ea8-ba41-67eb1d6956ec None admin] RBAC: Adding query filter params (name=service) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 21:59:36.812153 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21492c1d-ac95-4ea8-ba41-67eb1d6956ec None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:36.813533 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21492c1d-ac95-4ea8-ba41-67eb1d6956ec None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:36.820924 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-638574a2-2143-4ab0-997d-c9b465a9a155 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:36.827184 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 339/667] 10.222.0.52 () {56 vars in 1103 bytes} [Wed Jul 13 21:59:36 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 21:59:36.827184 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21492c1d-ac95-4ea8-ba41-67eb1d6956ec None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:36.864857 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48a1664-e79a-4f86-b328-d0ed3295a1e5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:36.865534 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48a1664-e79a-4f86-b328-d0ed3295a1e5 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:36.865814 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48a1664-e79a-4f86-b328-d0ed3295a1e5 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:36.866093 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b48a1664-e79a-4f86-b328-d0ed3295a1e5 None admin] PATH_INFO: `/v3/users` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:36.866956 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b48a1664-e79a-4f86-b328-d0ed3295a1e5 None admin] RBAC: Adding query filter params (domain_id=default, name=neutron) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 21:59:36.867483 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b48a1664-e79a-4f86-b328-d0ed3295a1e5 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:36.868730 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b48a1664-e79a-4f86-b328-d0ed3295a1e5 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:36.886900 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 329/668] 10.222.0.52 () {56 vars in 1139 bytes} [Wed Jul 13 21:59:36 2022] GET /identity/v3/users?domain_id=default&name=neutron => generated 374 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 21:59:36.887726 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48a1664-e79a-4f86-b328-d0ed3295a1e5 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:36.934845 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21492c1d-ac95-4ea8-ba41-67eb1d6956ec None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:36.960825 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:59:36.969956 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d7e405-f391-4074-b410-5dae3f9d6f38 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:36.970820 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d7e405-f391-4074-b410-5dae3f9d6f38 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:36.971098 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d7e405-f391-4074-b410-5dae3f9d6f38 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:36.971372 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44d7e405-f391-4074-b410-5dae3f9d6f38 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:36.982646 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d7e405-f391-4074-b410-5dae3f9d6f38 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:36.984115 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44d7e405-f391-4074-b410-5dae3f9d6f38 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:36.994620 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b48a1664-e79a-4f86-b328-d0ed3295a1e5 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:37.001290 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application [None req-44d7e405-f391-4074-b410-5dae3f9d6f38 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 21:59:37.001290 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 21:59:37.001290 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 21:59:37.001290 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 21:59:37.001290 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 21:59:37.001290 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 21:59:37.001290 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 21:59:37.001290 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 21:59:37.001290 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 21:59:37.001290 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 21:59:37.001290 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 21:59:37.001290 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 21:59:37.001290 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 21:59:37.001290 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 21:59:37.001290 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 21:59:37.002188 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 21:59:37.002188 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 21:59:37.002188 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 21:59:37.002188 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 21:59:37.002188 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 21:59:37.002188 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 21:59:37.002188 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 21:59:37.002188 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 21:59:37.002188 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application with Lock( Jul 13 21:59:37.002188 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 21:59:37.002188 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._enter() Jul 13 21:59:37.002188 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 21:59:37.002188 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 21:59:37.002188 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 21:59:37.002188 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.creator() Jul 13 21:59:37.002188 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 21:59:37.002188 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application created_value = creator( Jul 13 21:59:37.003144 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 21:59:37.003144 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 21:59:37.003144 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 21:59:37.003144 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Jul 13 21:59:37.003144 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 340/669] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Jul 13 21:59:36 2022] GET /identity/v3/domains/Default => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 21:59:37.003428 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d7e405-f391-4074-b410-5dae3f9d6f38 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:37.024303 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:59:37.038630 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8fc58a-d959-4b82-8166-696ac3cccd99 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:37.039539 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8fc58a-d959-4b82-8166-696ac3cccd99 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:37.040005 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8fc58a-d959-4b82-8166-696ac3cccd99 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:37.040499 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a8fc58a-d959-4b82-8166-696ac3cccd99 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:37.041627 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a8fc58a-d959-4b82-8166-696ac3cccd99 None admin] RBAC: Adding query filter params (role.id=70c34ea493484322911dab62b4a56e75, scope.project.id=417e3e2f1333481795a089bef2e24bea, user.id=e4d35d50e15d43d3afc59206b8165f30) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 21:59:37.042200 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a8fc58a-d959-4b82-8166-696ac3cccd99 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:37.043675 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a8fc58a-d959-4b82-8166-696ac3cccd99 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:37.050410 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 330/670] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Jul 13 21:59:36 2022] GET /identity/v3/role_assignments?user.id=e4d35d50e15d43d3afc59206b8165f30&scope.project.id=417e3e2f1333481795a089bef2e24bea&role.id=70c34ea493484322911dab62b4a56e75 => generated 608 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 21:59:37.059557 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8fc58a-d959-4b82-8166-696ac3cccd99 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:37.132880 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44d7e405-f391-4074-b410-5dae3f9d6f38 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:37.174953 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d41afa1-3bc9-4a8a-b240-f0817c7c5f0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:37.175886 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d41afa1-3bc9-4a8a-b240-f0817c7c5f0b None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:37.176165 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d41afa1-3bc9-4a8a-b240-f0817c7c5f0b None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:37.176533 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d41afa1-3bc9-4a8a-b240-f0817c7c5f0b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:37.189884 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d41afa1-3bc9-4a8a-b240-f0817c7c5f0b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:37.191551 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d41afa1-3bc9-4a8a-b240-f0817c7c5f0b None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:37.194022 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a8fc58a-d959-4b82-8166-696ac3cccd99 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:37.211750 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application [None req-7d41afa1-3bc9-4a8a-b240-f0817c7c5f0b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 21:59:37.211750 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 21:59:37.211750 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 21:59:37.211750 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 21:59:37.211750 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 21:59:37.211750 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 21:59:37.211750 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 21:59:37.211750 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 21:59:37.211750 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 21:59:37.211750 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 21:59:37.211750 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 21:59:37.211750 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 21:59:37.211750 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 21:59:37.211750 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 21:59:37.211750 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 21:59:37.212792 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 21:59:37.212792 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 21:59:37.212792 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 21:59:37.212792 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 21:59:37.212792 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 21:59:37.212792 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 21:59:37.212792 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 21:59:37.212792 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 21:59:37.212792 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application with Lock( Jul 13 21:59:37.212792 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 21:59:37.212792 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._enter() Jul 13 21:59:37.212792 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 21:59:37.212792 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 21:59:37.212792 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 21:59:37.212792 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.creator() Jul 13 21:59:37.212792 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 21:59:37.212792 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application created_value = creator( Jul 13 21:59:37.213687 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 21:59:37.213687 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 21:59:37.213687 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 21:59:37.213687 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Jul 13 21:59:37.213687 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 341/671] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Jul 13 21:59:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 210 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 21:59:37.213974 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d41afa1-3bc9-4a8a-b240-f0817c7c5f0b None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:37.240109 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3732f99-8a29-4c13-ac0e-1634b01c98bf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:37.240718 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3732f99-8a29-4c13-ac0e-1634b01c98bf None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:37.241142 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3732f99-8a29-4c13-ac0e-1634b01c98bf None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:37.241820 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3732f99-8a29-4c13-ac0e-1634b01c98bf None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:37.245111 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:59:37.305472 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:59:37.333996 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3732f99-8a29-4c13-ac0e-1634b01c98bf None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:37.335654 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3732f99-8a29-4c13-ac0e-1634b01c98bf None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:37.345491 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d41afa1-3bc9-4a8a-b240-f0817c7c5f0b None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:37.382873 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a722fe3-1beb-438b-b9e7-629c3c0322c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:37.383550 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a722fe3-1beb-438b-b9e7-629c3c0322c8 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:37.383832 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a722fe3-1beb-438b-b9e7-629c3c0322c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:37.384274 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a722fe3-1beb-438b-b9e7-629c3c0322c8 None admin] PATH_INFO: `/v3/domains` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:37.384853 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a722fe3-1beb-438b-b9e7-629c3c0322c8 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 21:59:37.385158 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a722fe3-1beb-438b-b9e7-629c3c0322c8 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:37.386735 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a722fe3-1beb-438b-b9e7-629c3c0322c8 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:37.399354 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 342/672] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Jul 13 21:59:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 21:59:37.400256 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a722fe3-1beb-438b-b9e7-629c3c0322c8 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:37.461772 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 331/673] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Jul 13 21:59:37 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 411 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 13 21:59:37.463364 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3732f99-8a29-4c13-ac0e-1634b01c98bf None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:37.507921 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:59:37.524119 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a722fe3-1beb-438b-b9e7-629c3c0322c8 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:37.560530 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:59:37.572389 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78d032e-5ed6-4980-8715-f978588ecf71 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:37.573388 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78d032e-5ed6-4980-8715-f978588ecf71 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:37.574386 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78d032e-5ed6-4980-8715-f978588ecf71 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:37.574839 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f78d032e-5ed6-4980-8715-f978588ecf71 None admin] PATH_INFO: `/v3/domains` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:37.576264 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f78d032e-5ed6-4980-8715-f978588ecf71 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 21:59:37.576825 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f78d032e-5ed6-4980-8715-f978588ecf71 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:37.578975 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f78d032e-5ed6-4980-8715-f978588ecf71 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:37.588495 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3732f99-8a29-4c13-ac0e-1634b01c98bf None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:37.600736 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 343/674] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Jul 13 21:59:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 196 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 21:59:37.600736 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78d032e-5ed6-4980-8715-f978588ecf71 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:37.636401 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6a3d16-7de6-484c-9306-14f41c7274de None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:37.637584 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6a3d16-7de6-484c-9306-14f41c7274de None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:37.638056 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6a3d16-7de6-484c-9306-14f41c7274de None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:37.638562 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db6a3d16-7de6-484c-9306-14f41c7274de None admin] PATH_INFO: `/v3/projects/service` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:37.647861 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db6a3d16-7de6-484c-9306-14f41c7274de None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:37.650263 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db6a3d16-7de6-484c-9306-14f41c7274de None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:37.668241 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application [None req-db6a3d16-7de6-484c-9306-14f41c7274de None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 13 21:59:37.668241 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 21:59:37.668241 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 21:59:37.668241 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 21:59:37.668241 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 21:59:37.668241 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 21:59:37.668241 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 21:59:37.668241 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 21:59:37.668241 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 21:59:37.668241 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 21:59:37.668241 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 21:59:37.668241 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 21:59:37.668241 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 13 21:59:37.668241 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 13 21:59:37.668241 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 13 21:59:37.669146 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 13 21:59:37.669146 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 21:59:37.669146 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 21:59:37.669146 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 21:59:37.669146 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 21:59:37.669146 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 21:59:37.669146 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 21:59:37.669146 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 21:59:37.669146 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application with Lock( Jul 13 21:59:37.669146 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 21:59:37.669146 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._enter() Jul 13 21:59:37.669146 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 21:59:37.669146 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 21:59:37.669146 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 21:59:37.669146 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.creator() Jul 13 21:59:37.669146 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 21:59:37.669146 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application created_value = creator( Jul 13 21:59:37.670034 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 13 21:59:37.670034 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 13 21:59:37.670034 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 13 21:59:37.670034 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 13 21:59:37.670034 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 13 21:59:37.670034 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 13 21:59:37.670034 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 13 21:59:37.670034 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Jul 13 21:59:37.672364 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 332/675] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Jul 13 21:59:37 2022] GET /identity/v3/projects/service => generated 88 bytes in 209 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 21:59:37.673382 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6a3d16-7de6-484c-9306-14f41c7274de None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:37.710636 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f78d032e-5ed6-4980-8715-f978588ecf71 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:37.747959 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c3866c-f6c7-409f-aced-086c6bc30f21 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:37.748844 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c3866c-f6c7-409f-aced-086c6bc30f21 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:37.749222 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c3866c-f6c7-409f-aced-086c6bc30f21 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:37.749745 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40c3866c-f6c7-409f-aced-086c6bc30f21 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:37.764423 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c3866c-f6c7-409f-aced-086c6bc30f21 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:37.766174 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40c3866c-f6c7-409f-aced-086c6bc30f21 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:37.788243 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application [None req-40c3866c-f6c7-409f-aced-086c6bc30f21 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 21:59:37.788243 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 21:59:37.788243 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 21:59:37.788243 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 21:59:37.788243 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 21:59:37.788243 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 21:59:37.788243 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 21:59:37.788243 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 21:59:37.788243 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 21:59:37.788243 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 21:59:37.788243 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 21:59:37.788243 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 21:59:37.788243 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 21:59:37.788243 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 21:59:37.788243 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 21:59:37.789172 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 21:59:37.789172 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 21:59:37.789172 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 21:59:37.789172 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 21:59:37.789172 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 21:59:37.789172 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 21:59:37.789172 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 21:59:37.789172 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 21:59:37.789172 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application with Lock( Jul 13 21:59:37.789172 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 21:59:37.789172 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._enter() Jul 13 21:59:37.789172 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 21:59:37.789172 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 21:59:37.789172 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 21:59:37.789172 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.creator() Jul 13 21:59:37.789172 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 21:59:37.789172 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application created_value = creator( Jul 13 21:59:37.790072 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 21:59:37.790072 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 21:59:37.790072 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 21:59:37.790072 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Jul 13 21:59:37.791365 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:59:37.797569 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db6a3d16-7de6-484c-9306-14f41c7274de None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:37.798846 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 344/676] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Jul 13 21:59:37 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 21:59:37.799504 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c3866c-f6c7-409f-aced-086c6bc30f21 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:37.836681 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4378d94-c01d-42ee-853d-fc7f28a3b33a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:37.837822 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4378d94-c01d-42ee-853d-fc7f28a3b33a None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:37.838822 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4378d94-c01d-42ee-853d-fc7f28a3b33a None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:37.838822 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4378d94-c01d-42ee-853d-fc7f28a3b33a None admin] PATH_INFO: `/v3/users/swift` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:37.845749 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:59:37.847882 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4378d94-c01d-42ee-853d-fc7f28a3b33a None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:37.849449 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4378d94-c01d-42ee-853d-fc7f28a3b33a None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:37.867288 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application [None req-e4378d94-c01d-42ee-853d-fc7f28a3b33a None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 13 21:59:37.867288 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 21:59:37.867288 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 21:59:37.867288 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 21:59:37.867288 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 21:59:37.867288 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 21:59:37.867288 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 21:59:37.867288 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 21:59:37.867288 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 21:59:37.867288 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 21:59:37.867288 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 21:59:37.867288 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 21:59:37.867288 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 13 21:59:37.867288 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 13 21:59:37.867288 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 13 21:59:37.867288 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 13 21:59:37.868655 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 21:59:37.868655 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 21:59:37.868655 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 13 21:59:37.868655 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 21:59:37.868655 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 13 21:59:37.868655 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 21:59:37.868655 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 21:59:37.868655 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 21:59:37.868655 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 21:59:37.868655 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 21:59:37.868655 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 21:59:37.868655 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application with Lock( Jul 13 21:59:37.868655 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 21:59:37.868655 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._enter() Jul 13 21:59:37.868655 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 21:59:37.868655 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 21:59:37.868655 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 21:59:37.868655 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.creator() Jul 13 21:59:37.869592 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 21:59:37.869592 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application created_value = creator( Jul 13 21:59:37.869592 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 13 21:59:37.869592 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 13 21:59:37.869592 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 13 21:59:37.869592 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 13 21:59:37.869592 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 13 21:59:37.869592 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 13 21:59:37.869592 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 13 21:59:37.869592 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Jul 13 21:59:37.873945 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 333/677] 10.222.0.52 () {56 vars in 1096 bytes} [Wed Jul 13 21:59:37 2022] GET /identity/v3/users/swift => generated 83 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 21:59:37.874947 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4378d94-c01d-42ee-853d-fc7f28a3b33a None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:37.910576 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40c3866c-f6c7-409f-aced-086c6bc30f21 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:37.961944 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81274403-e259-4d5d-993c-b2cbf4507b5c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:37.963324 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81274403-e259-4d5d-993c-b2cbf4507b5c None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:37.966597 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81274403-e259-4d5d-993c-b2cbf4507b5c None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:37.966597 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81274403-e259-4d5d-993c-b2cbf4507b5c None admin] PATH_INFO: `/v3/projects` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:37.966597 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81274403-e259-4d5d-993c-b2cbf4507b5c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 21:59:37.972517 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81274403-e259-4d5d-993c-b2cbf4507b5c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:37.974009 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81274403-e259-4d5d-993c-b2cbf4507b5c None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:37.997631 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 345/678] 10.222.0.52 () {56 vars in 1148 bytes} [Wed Jul 13 21:59:37 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 21:59:37.998552 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81274403-e259-4d5d-993c-b2cbf4507b5c None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:38.023402 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4378d94-c01d-42ee-853d-fc7f28a3b33a None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:38.067310 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5f59d7-4b1f-4df4-96e3-72bb4caf5324 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:38.068019 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5f59d7-4b1f-4df4-96e3-72bb4caf5324 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:38.068301 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5f59d7-4b1f-4df4-96e3-72bb4caf5324 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:38.068601 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e5f59d7-4b1f-4df4-96e3-72bb4caf5324 None admin] PATH_INFO: `/v3/domains` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:38.069756 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e5f59d7-4b1f-4df4-96e3-72bb4caf5324 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 21:59:38.070068 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e5f59d7-4b1f-4df4-96e3-72bb4caf5324 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:38.072232 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e5f59d7-4b1f-4df4-96e3-72bb4caf5324 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:38.088246 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 334/679] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Jul 13 21:59:37 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 213 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 21:59:38.089148 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5f59d7-4b1f-4df4-96e3-72bb4caf5324 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:38.110697 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:59:38.157944 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:59:38.162510 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81274403-e259-4d5d-993c-b2cbf4507b5c None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:38.226067 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5408c58d-7afa-4649-beb1-b4f9277f7905 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:38.226871 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5408c58d-7afa-4649-beb1-b4f9277f7905 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:38.227220 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5408c58d-7afa-4649-beb1-b4f9277f7905 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:38.227567 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5408c58d-7afa-4649-beb1-b4f9277f7905 None admin] PATH_INFO: `/v3/users` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:38.228598 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5408c58d-7afa-4649-beb1-b4f9277f7905 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 21:59:38.230608 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5408c58d-7afa-4649-beb1-b4f9277f7905 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:38.230608 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5408c58d-7afa-4649-beb1-b4f9277f7905 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:38.255884 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 346/680] 10.222.0.52 () {56 vars in 1135 bytes} [Wed Jul 13 21:59:37 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 258 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 21:59:38.256657 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5408c58d-7afa-4649-beb1-b4f9277f7905 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:38.264256 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e5f59d7-4b1f-4df4-96e3-72bb4caf5324 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:38.318049 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e499adbb-e12f-40ab-9c6c-17311b75e3c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:38.318768 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e499adbb-e12f-40ab-9c6c-17311b75e3c7 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:38.319050 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e499adbb-e12f-40ab-9c6c-17311b75e3c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:38.319322 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e499adbb-e12f-40ab-9c6c-17311b75e3c7 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:38.320176 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e499adbb-e12f-40ab-9c6c-17311b75e3c7 None admin] RBAC: Adding query filter params (role.id=70c34ea493484322911dab62b4a56e75, scope.project.id=417e3e2f1333481795a089bef2e24bea, user.id=bc77e1ea1ada4700b809beb1c980c5e1) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 21:59:38.320507 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e499adbb-e12f-40ab-9c6c-17311b75e3c7 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:38.321995 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e499adbb-e12f-40ab-9c6c-17311b75e3c7 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:38.338049 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 335/681] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Jul 13 21:59:38 2022] GET /identity/v3/role_assignments?user.id=bc77e1ea1ada4700b809beb1c980c5e1&scope.project.id=417e3e2f1333481795a089bef2e24bea&role.id=70c34ea493484322911dab62b4a56e75 => generated 608 bytes in 241 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 21:59:38.338049 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b93602e-6764-49d4-870b-abe07f6f28a5 None None] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:38.338049 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b93602e-6764-49d4-870b-abe07f6f28a5 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:38.338049 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b93602e-6764-49d4-870b-abe07f6f28a5 None None] PATH_INFO: `/` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:38.338049 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 336/682] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jul 13 21:59:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 21:59:38.338049 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b93602e-6764-49d4-870b-abe07f6f28a5 None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:38.409994 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5408c58d-7afa-4649-beb1-b4f9277f7905 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:38.485084 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d45081d7-f6bf-4148-b5e0-4dd4fc2d6979 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:38.486297 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45081d7-f6bf-4148-b5e0-4dd4fc2d6979 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:38.486847 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45081d7-f6bf-4148-b5e0-4dd4fc2d6979 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:38.487320 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d45081d7-f6bf-4148-b5e0-4dd4fc2d6979 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:38.500364 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45081d7-f6bf-4148-b5e0-4dd4fc2d6979 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:38.501756 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d45081d7-f6bf-4148-b5e0-4dd4fc2d6979 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:38.513596 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:59:38.518466 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application [None req-d45081d7-f6bf-4148-b5e0-4dd4fc2d6979 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 13 21:59:38.518466 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 21:59:38.518466 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 21:59:38.518466 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 21:59:38.518466 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 21:59:38.518466 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 21:59:38.518466 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 21:59:38.518466 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 21:59:38.518466 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 21:59:38.518466 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 21:59:38.518466 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 21:59:38.518466 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 21:59:38.518466 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 13 21:59:38.518466 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 13 21:59:38.518466 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 13 21:59:38.519415 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 13 21:59:38.519415 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 21:59:38.519415 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 21:59:38.519415 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 21:59:38.519415 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 21:59:38.519415 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 21:59:38.519415 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 21:59:38.519415 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 21:59:38.519415 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application with Lock( Jul 13 21:59:38.519415 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 21:59:38.519415 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._enter() Jul 13 21:59:38.519415 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 21:59:38.519415 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 21:59:38.519415 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 21:59:38.519415 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.creator() Jul 13 21:59:38.519415 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 21:59:38.519415 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application created_value = creator( Jul 13 21:59:38.520607 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 13 21:59:38.520607 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 13 21:59:38.520607 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 13 21:59:38.520607 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 13 21:59:38.520607 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 13 21:59:38.520607 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 13 21:59:38.520607 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 13 21:59:38.520607 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Jul 13 21:59:38.521044 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 347/683] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Jul 13 21:59:38 2022] GET /identity/v3/projects/service => generated 88 bytes in 263 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 21:59:38.524158 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead0df38-d44f-426b-9b85-8e1cbad6aac3 None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:38.524158 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead0df38-d44f-426b-9b85-8e1cbad6aac3 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:38.524158 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ead0df38-d44f-426b-9b85-8e1cbad6aac3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:38.544924 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-ead0df38-d44f-426b-9b85-8e1cbad6aac3 None None] MFA Rules not processed for user `cc99c93da5074cf5a48bc2aa9a92c5e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:59:38.546127 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-ead0df38-d44f-426b-9b85-8e1cbad6aac3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:59:38.571754 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:59:38.572462 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b93602e-6764-49d4-870b-abe07f6f28a5 None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:38.613453 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f0e9e6-3932-4865-adbd-bd1867648444 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:38.614982 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f0e9e6-3932-4865-adbd-bd1867648444 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:38.615474 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f0e9e6-3932-4865-adbd-bd1867648444 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:38.615954 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f0e9e6-3932-4865-adbd-bd1867648444 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:38.646527 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 348/684] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jul 13 21:59:38 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 120 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 13 21:59:38.646527 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead0df38-d44f-426b-9b85-8e1cbad6aac3 None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:38.709009 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f0e9e6-3932-4865-adbd-bd1867648444 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:38.710874 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2f0e9e6-3932-4865-adbd-bd1867648444 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:38.751219 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ead0df38-d44f-426b-9b85-8e1cbad6aac3 None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:38.788817 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e046c962-cc94-4ee7-83d7-217c71a316d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:38.789425 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e046c962-cc94-4ee7-83d7-217c71a316d9 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:38.789697 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e046c962-cc94-4ee7-83d7-217c71a316d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:38.790027 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e046c962-cc94-4ee7-83d7-217c71a316d9 None admin] PATH_INFO: `/v3/projects` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:38.791159 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e046c962-cc94-4ee7-83d7-217c71a316d9 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 21:59:38.791361 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e046c962-cc94-4ee7-83d7-217c71a316d9 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:38.792924 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:59:38.793515 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e046c962-cc94-4ee7-83d7-217c71a316d9 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:38.806931 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 349/685] 10.222.0.52 () {56 vars in 1148 bytes} [Wed Jul 13 21:59:38 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 21:59:38.808148 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e046c962-cc94-4ee7-83d7-217c71a316d9 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:38.816862 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 337/686] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Jul 13 21:59:38 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 483 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 13 21:59:38.818033 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f0e9e6-3932-4865-adbd-bd1867648444 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:38.820422 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:59:38.920732 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e046c962-cc94-4ee7-83d7-217c71a316d9 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:38.924305 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2f0e9e6-3932-4865-adbd-bd1867648444 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:38.964103 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b266ee-06fe-4eca-a920-73e505db2c89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:38.964917 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b266ee-06fe-4eca-a920-73e505db2c89 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:38.965269 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b266ee-06fe-4eca-a920-73e505db2c89 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:38.965506 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74b266ee-06fe-4eca-a920-73e505db2c89 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:38.967295 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344410a3-c967-4087-a55c-e20306e8cff0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:38.968780 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344410a3-c967-4087-a55c-e20306e8cff0 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:38.969272 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344410a3-c967-4087-a55c-e20306e8cff0 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:38.969738 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-344410a3-c967-4087-a55c-e20306e8cff0 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:38.970949 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344410a3-c967-4087-a55c-e20306e8cff0 None admin] RBAC: Adding query filter params (role.id=70c34ea493484322911dab62b4a56e75, scope.project.id=417e3e2f1333481795a089bef2e24bea, user.id=847ec9a1bd4d4dec804318a5c845895a) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 21:59:38.971557 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344410a3-c967-4087-a55c-e20306e8cff0 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:38.973592 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-344410a3-c967-4087-a55c-e20306e8cff0 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:38.976984 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74b266ee-06fe-4eca-a920-73e505db2c89 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 21:59:38.981070 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 338/687] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Jul 13 21:59:38 2022] GET /identity/v3/role_assignments?user.id=847ec9a1bd4d4dec804318a5c845895a&scope.project.id=417e3e2f1333481795a089bef2e24bea&role.id=70c34ea493484322911dab62b4a56e75 => generated 608 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 21:59:38.982185 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344410a3-c967-4087-a55c-e20306e8cff0 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:38.985345 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application [None req-74b266ee-06fe-4eca-a920-73e505db2c89 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 13 21:59:38.985345 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 21:59:38.985345 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 21:59:38.985345 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 21:59:38.985345 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 21:59:38.985345 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 21:59:38.985345 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 21:59:38.985345 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 21:59:38.985345 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 21:59:38.985345 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 21:59:38.985345 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 21:59:38.985345 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 21:59:38.985345 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 21:59:38.985345 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 21:59:38.985345 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 21:59:38.985345 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 21:59:38.986318 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 21:59:38.986318 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 21:59:38.986318 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 21:59:38.986318 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 21:59:38.986318 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 21:59:38.986318 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 21:59:38.986318 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 21:59:38.986318 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 21:59:38.986318 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 21:59:38.986318 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 21:59:38.986318 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 21:59:38.986318 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application with Lock( Jul 13 21:59:38.986318 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 21:59:38.986318 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._enter() Jul 13 21:59:38.986318 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 21:59:38.986318 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 21:59:38.993886 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 21:59:38.993886 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.creator() Jul 13 21:59:38.993886 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 21:59:38.993886 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application created_value = creator( Jul 13 21:59:38.993886 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 21:59:38.993886 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 21:59:38.993886 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 21:59:38.993886 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 21:59:38.993886 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 21:59:38.993886 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 21:59:38.993886 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 13 21:59:38.993886 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Jul 13 21:59:38.993886 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 350/688] 10.222.0.52 () {56 vars in 1096 bytes} [Wed Jul 13 21:59:38 2022] GET /identity/v3/roles/admin => generated 83 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 21:59:38.996487 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b266ee-06fe-4eca-a920-73e505db2c89 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:39.079406 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:59:39.120173 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74b266ee-06fe-4eca-a920-73e505db2c89 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:39.123006 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:59:39.176888 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b1d138-cbd2-48b8-a8e1-a0176f637d15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:39.177980 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b1d138-cbd2-48b8-a8e1-a0176f637d15 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:39.178455 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b1d138-cbd2-48b8-a8e1-a0176f637d15 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:39.178957 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3b1d138-cbd2-48b8-a8e1-a0176f637d15 None admin] PATH_INFO: `/v3/roles` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:39.180323 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b1d138-cbd2-48b8-a8e1-a0176f637d15 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 21:59:39.180905 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b1d138-cbd2-48b8-a8e1-a0176f637d15 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:39.182890 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3b1d138-cbd2-48b8-a8e1-a0176f637d15 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:39.198668 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 351/689] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Jul 13 21:59:38 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 21:59:39.201109 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b1d138-cbd2-48b8-a8e1-a0176f637d15 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:39.221639 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-344410a3-c967-4087-a55c-e20306e8cff0 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:39.266208 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dcb1408-c50b-4532-9870-7b81778b4989 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:39.267586 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcb1408-c50b-4532-9870-7b81778b4989 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:39.267866 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcb1408-c50b-4532-9870-7b81778b4989 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:39.268285 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dcb1408-c50b-4532-9870-7b81778b4989 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:39.280919 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dcb1408-c50b-4532-9870-7b81778b4989 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:39.283070 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dcb1408-c50b-4532-9870-7b81778b4989 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:39.300723 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application [None req-5dcb1408-c50b-4532-9870-7b81778b4989 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 21:59:39.300723 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 21:59:39.300723 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 21:59:39.300723 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 21:59:39.300723 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 21:59:39.300723 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 21:59:39.300723 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 21:59:39.300723 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 21:59:39.300723 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 21:59:39.300723 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 21:59:39.300723 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 21:59:39.300723 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 21:59:39.300723 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 21:59:39.300723 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 21:59:39.300723 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 21:59:39.301668 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 21:59:39.301668 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 21:59:39.301668 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 21:59:39.301668 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 21:59:39.301668 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 21:59:39.301668 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 21:59:39.301668 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 21:59:39.301668 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 21:59:39.301668 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application with Lock( Jul 13 21:59:39.301668 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 21:59:39.301668 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._enter() Jul 13 21:59:39.301668 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 21:59:39.301668 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 21:59:39.301668 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 21:59:39.301668 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.creator() Jul 13 21:59:39.301668 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 21:59:39.301668 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application created_value = creator( Jul 13 21:59:39.302610 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 21:59:39.302610 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 21:59:39.302610 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 21:59:39.302610 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Jul 13 21:59:39.302610 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 339/690] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Jul 13 21:59:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 320 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 21:59:39.303629 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb3a71d-bb37-4fc1-a376-7031d1b70531 None None] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:39.303940 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb3a71d-bb37-4fc1-a376-7031d1b70531 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:39.304196 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb3a71d-bb37-4fc1-a376-7031d1b70531 None None] PATH_INFO: `/` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:39.309590 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 340/691] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jul 13 21:59:39 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 21:59:39.309590 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb3a71d-bb37-4fc1-a376-7031d1b70531 None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:39.331493 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3b1d138-cbd2-48b8-a8e1-a0176f637d15 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:39.370213 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:59:39.377372 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34e5184b-859a-46ad-a51b-5fec46d90365 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:39.378620 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e5184b-859a-46ad-a51b-5fec46d90365 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:39.379137 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e5184b-859a-46ad-a51b-5fec46d90365 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:39.379644 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34e5184b-859a-46ad-a51b-5fec46d90365 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:39.392054 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e5184b-859a-46ad-a51b-5fec46d90365 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:39.393797 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34e5184b-859a-46ad-a51b-5fec46d90365 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:39.412240 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application [None req-34e5184b-859a-46ad-a51b-5fec46d90365 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 21:59:39.412240 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 21:59:39.412240 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 21:59:39.412240 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 21:59:39.412240 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 21:59:39.412240 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 21:59:39.412240 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 21:59:39.412240 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 21:59:39.412240 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 21:59:39.412240 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 21:59:39.412240 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 21:59:39.412240 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 21:59:39.412240 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 21:59:39.412240 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 21:59:39.412240 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 21:59:39.413197 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 21:59:39.413197 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 21:59:39.413197 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 21:59:39.413197 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 21:59:39.413197 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 21:59:39.413197 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 21:59:39.413197 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 21:59:39.413197 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 21:59:39.413197 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application with Lock( Jul 13 21:59:39.413197 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 21:59:39.413197 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._enter() Jul 13 21:59:39.413197 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 21:59:39.413197 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 21:59:39.413197 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 21:59:39.413197 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.creator() Jul 13 21:59:39.413197 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 21:59:39.413197 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application created_value = creator( Jul 13 21:59:39.414093 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 21:59:39.414093 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 21:59:39.414093 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 21:59:39.414093 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Jul 13 21:59:39.415268 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:59:39.417365 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 352/692] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Jul 13 21:59:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 217 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 21:59:39.419139 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6539cbf0-3a47-4b85-bfc1-506980879400 None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:39.419614 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6539cbf0-3a47-4b85-bfc1-506980879400 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:39.421103 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6539cbf0-3a47-4b85-bfc1-506980879400 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:39.422478 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb3a71d-bb37-4fc1-a376-7031d1b70531 None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:39.440407 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-6539cbf0-3a47-4b85-bfc1-506980879400 None None] MFA Rules not processed for user `cc99c93da5074cf5a48bc2aa9a92c5e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:59:39.441921 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-6539cbf0-3a47-4b85-bfc1-506980879400 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:59:39.459765 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389744ec-150a-4d1b-aa2e-0fd1a88c4b2e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:39.460710 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389744ec-150a-4d1b-aa2e-0fd1a88c4b2e None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:39.461394 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389744ec-150a-4d1b-aa2e-0fd1a88c4b2e None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:39.461708 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389744ec-150a-4d1b-aa2e-0fd1a88c4b2e None admin] PATH_INFO: `/v3/domains` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:39.462662 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389744ec-150a-4d1b-aa2e-0fd1a88c4b2e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 21:59:39.463070 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389744ec-150a-4d1b-aa2e-0fd1a88c4b2e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:39.464552 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-389744ec-150a-4d1b-aa2e-0fd1a88c4b2e None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:39.478438 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 341/693] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Jul 13 21:59:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 21:59:39.479379 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389744ec-150a-4d1b-aa2e-0fd1a88c4b2e None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:39.522604 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 353/694] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jul 13 21:59:39 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 13 21:59:39.523761 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6539cbf0-3a47-4b85-bfc1-506980879400 None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:39.587016 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-389744ec-150a-4d1b-aa2e-0fd1a88c4b2e None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:39.626402 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b0bf1a-2985-4cd3-b2e6-1901b201c40f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:39.627207 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b0bf1a-2985-4cd3-b2e6-1901b201c40f None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:39.627597 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b0bf1a-2985-4cd3-b2e6-1901b201c40f None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:39.627968 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17b0bf1a-2985-4cd3-b2e6-1901b201c40f None admin] PATH_INFO: `/v3/domains` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:39.628815 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b0bf1a-2985-4cd3-b2e6-1901b201c40f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 21:59:39.629220 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b0bf1a-2985-4cd3-b2e6-1901b201c40f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:39.630654 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17b0bf1a-2985-4cd3-b2e6-1901b201c40f None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:39.644153 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 342/695] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Jul 13 21:59:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 21:59:39.645277 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b0bf1a-2985-4cd3-b2e6-1901b201c40f None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:39.647126 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6539cbf0-3a47-4b85-bfc1-506980879400 None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:39.665710 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:59:39.672529 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:59:39.686643 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319152ea-13a2-4bfe-b1f0-e49d08200a6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:39.687836 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319152ea-13a2-4bfe-b1f0-e49d08200a6e None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:39.688360 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319152ea-13a2-4bfe-b1f0-e49d08200a6e None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:39.688870 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-319152ea-13a2-4bfe-b1f0-e49d08200a6e None admin] PATH_INFO: `/v3/projects/service` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:39.696399 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319152ea-13a2-4bfe-b1f0-e49d08200a6e None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:39.698207 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-319152ea-13a2-4bfe-b1f0-e49d08200a6e None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:39.713688 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application [None req-319152ea-13a2-4bfe-b1f0-e49d08200a6e None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 13 21:59:39.713688 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 21:59:39.713688 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 21:59:39.713688 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 21:59:39.713688 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 21:59:39.713688 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 21:59:39.713688 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 21:59:39.713688 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 21:59:39.713688 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 21:59:39.713688 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 21:59:39.713688 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 21:59:39.713688 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 21:59:39.713688 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 13 21:59:39.713688 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 13 21:59:39.713688 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 13 21:59:39.714645 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 13 21:59:39.714645 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 21:59:39.714645 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 21:59:39.714645 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 21:59:39.714645 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 21:59:39.714645 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 21:59:39.714645 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 21:59:39.714645 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 21:59:39.714645 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application with Lock( Jul 13 21:59:39.714645 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 21:59:39.714645 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._enter() Jul 13 21:59:39.714645 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 21:59:39.714645 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 21:59:39.714645 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 21:59:39.714645 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.creator() Jul 13 21:59:39.714645 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 21:59:39.714645 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application created_value = creator( Jul 13 21:59:39.715526 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 13 21:59:39.715526 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 13 21:59:39.715526 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 13 21:59:39.715526 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 13 21:59:39.715526 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 13 21:59:39.715526 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 13 21:59:39.715526 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 13 21:59:39.715526 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Jul 13 21:59:39.718263 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 354/696] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Jul 13 21:59:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 21:59:39.719354 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319152ea-13a2-4bfe-b1f0-e49d08200a6e None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:39.766058 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17b0bf1a-2985-4cd3-b2e6-1901b201c40f None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:39.808365 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de82b12-6d40-4edd-8a42-3574627c86bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:39.809185 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de82b12-6d40-4edd-8a42-3574627c86bb None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:39.809463 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de82b12-6d40-4edd-8a42-3574627c86bb None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:39.809738 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de82b12-6d40-4edd-8a42-3574627c86bb None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:39.823600 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-319152ea-13a2-4bfe-b1f0-e49d08200a6e None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:39.824336 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de82b12-6d40-4edd-8a42-3574627c86bb None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:39.825959 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de82b12-6d40-4edd-8a42-3574627c86bb None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:39.849073 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application [None req-9de82b12-6d40-4edd-8a42-3574627c86bb None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 21:59:39.849073 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 21:59:39.849073 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 21:59:39.849073 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 21:59:39.849073 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 21:59:39.849073 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 21:59:39.849073 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 21:59:39.849073 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 21:59:39.849073 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 21:59:39.849073 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 21:59:39.849073 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 21:59:39.849073 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 21:59:39.849073 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 21:59:39.849073 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 21:59:39.849073 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 21:59:39.850002 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 21:59:39.850002 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 21:59:39.850002 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 21:59:39.850002 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 21:59:39.850002 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 21:59:39.850002 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 21:59:39.850002 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 21:59:39.850002 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 21:59:39.850002 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application with Lock( Jul 13 21:59:39.850002 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 21:59:39.850002 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._enter() Jul 13 21:59:39.850002 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 21:59:39.850002 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 21:59:39.850002 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 21:59:39.850002 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.creator() Jul 13 21:59:39.850002 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 21:59:39.850002 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application created_value = creator( Jul 13 21:59:39.850932 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 21:59:39.850932 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 21:59:39.850932 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 21:59:39.850932 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Jul 13 21:59:39.854562 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 343/697] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Jul 13 21:59:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 21:59:39.855668 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de82b12-6d40-4edd-8a42-3574627c86bb None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:39.871975 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f60c59-738c-4e43-a0c7-203fbe654ca0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:39.872784 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f60c59-738c-4e43-a0c7-203fbe654ca0 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:39.873058 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f60c59-738c-4e43-a0c7-203fbe654ca0 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:39.873324 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11f60c59-738c-4e43-a0c7-203fbe654ca0 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:39.884753 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f60c59-738c-4e43-a0c7-203fbe654ca0 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:39.887348 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11f60c59-738c-4e43-a0c7-203fbe654ca0 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:39.907294 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application [None req-11f60c59-738c-4e43-a0c7-203fbe654ca0 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 13 21:59:39.907294 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 21:59:39.907294 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 21:59:39.907294 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 21:59:39.907294 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 21:59:39.907294 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 21:59:39.907294 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 21:59:39.907294 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 21:59:39.907294 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 21:59:39.907294 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 21:59:39.907294 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 21:59:39.907294 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 21:59:39.907294 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 13 21:59:39.907294 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 13 21:59:39.907294 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 13 21:59:39.907294 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 13 21:59:39.908309 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 21:59:39.908309 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 21:59:39.908309 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 13 21:59:39.908309 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 21:59:39.908309 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 13 21:59:39.908309 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 21:59:39.908309 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 21:59:39.908309 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 21:59:39.908309 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 21:59:39.908309 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 21:59:39.908309 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 21:59:39.908309 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application with Lock( Jul 13 21:59:39.908309 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 21:59:39.908309 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._enter() Jul 13 21:59:39.908309 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 21:59:39.908309 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 21:59:39.908309 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 21:59:39.908309 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.creator() Jul 13 21:59:39.909267 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 21:59:39.909267 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application created_value = creator( Jul 13 21:59:39.909267 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 13 21:59:39.909267 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 13 21:59:39.909267 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 13 21:59:39.909267 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 13 21:59:39.909267 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 13 21:59:39.909267 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 13 21:59:39.909267 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 13 21:59:39.909267 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Jul 13 21:59:39.911763 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 355/698] 10.222.0.52 () {56 vars in 1093 bytes} [Wed Jul 13 21:59:39 2022] GET /identity/v3/users/nova => generated 82 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 21:59:39.913451 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f60c59-738c-4e43-a0c7-203fbe654ca0 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:39.932706 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:59:39.962766 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:59:39.977457 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de82b12-6d40-4edd-8a42-3574627c86bb None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:40.021891 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981b3eb1-7980-49fa-a3a8-87a290348d88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:40.022902 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981b3eb1-7980-49fa-a3a8-87a290348d88 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:40.023329 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981b3eb1-7980-49fa-a3a8-87a290348d88 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:40.023746 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-981b3eb1-7980-49fa-a3a8-87a290348d88 None admin] PATH_INFO: `/v3/projects` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:40.025157 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981b3eb1-7980-49fa-a3a8-87a290348d88 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 21:59:40.025645 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981b3eb1-7980-49fa-a3a8-87a290348d88 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:40.027708 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-981b3eb1-7980-49fa-a3a8-87a290348d88 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:40.038874 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11f60c59-738c-4e43-a0c7-203fbe654ca0 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:40.044193 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 344/699] 10.222.0.52 () {56 vars in 1148 bytes} [Wed Jul 13 21:59:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 21:59:40.045286 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981b3eb1-7980-49fa-a3a8-87a290348d88 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:40.082525 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2d2095-b576-45f3-98b4-61aaf751ec34 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:40.083488 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2d2095-b576-45f3-98b4-61aaf751ec34 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:40.084186 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2d2095-b576-45f3-98b4-61aaf751ec34 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:40.084612 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a2d2095-b576-45f3-98b4-61aaf751ec34 None admin] PATH_INFO: `/v3/domains` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:40.086376 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2d2095-b576-45f3-98b4-61aaf751ec34 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 21:59:40.086871 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2d2095-b576-45f3-98b4-61aaf751ec34 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:40.088470 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a2d2095-b576-45f3-98b4-61aaf751ec34 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:40.109333 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 356/700] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Jul 13 21:59:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 192 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 21:59:40.109333 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2d2095-b576-45f3-98b4-61aaf751ec34 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:40.161603 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-981b3eb1-7980-49fa-a3a8-87a290348d88 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:40.198399 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:59:40.211405 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297629a0-176e-4623-9ee8-e0447f450afc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:40.212727 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297629a0-176e-4623-9ee8-e0447f450afc None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:40.213590 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297629a0-176e-4623-9ee8-e0447f450afc None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:40.214087 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-297629a0-176e-4623-9ee8-e0447f450afc None admin] PATH_INFO: `/v3/users` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:40.215167 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297629a0-176e-4623-9ee8-e0447f450afc None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 21:59:40.215699 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297629a0-176e-4623-9ee8-e0447f450afc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:40.218122 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-297629a0-176e-4623-9ee8-e0447f450afc None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:40.219998 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:59:40.232655 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a2d2095-b576-45f3-98b4-61aaf751ec34 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:40.244685 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 345/701] 10.222.0.52 () {56 vars in 1133 bytes} [Wed Jul 13 21:59:40 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 21:59:40.245847 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297629a0-176e-4623-9ee8-e0447f450afc None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:40.276996 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5f370e-4008-4a2b-8ff8-38d11507d3c9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:40.277691 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5f370e-4008-4a2b-8ff8-38d11507d3c9 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:40.277967 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5f370e-4008-4a2b-8ff8-38d11507d3c9 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:40.278239 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c5f370e-4008-4a2b-8ff8-38d11507d3c9 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:40.279106 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c5f370e-4008-4a2b-8ff8-38d11507d3c9 None admin] RBAC: Adding query filter params (role.id=70c34ea493484322911dab62b4a56e75, scope.project.id=417e3e2f1333481795a089bef2e24bea, user.id=1eb1525233ba48c99b15d4cb75db4822) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 21:59:40.279495 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c5f370e-4008-4a2b-8ff8-38d11507d3c9 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:40.280856 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c5f370e-4008-4a2b-8ff8-38d11507d3c9 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:40.287521 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 357/702] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Jul 13 21:59:40 2022] GET /identity/v3/role_assignments?user.id=1eb1525233ba48c99b15d4cb75db4822&scope.project.id=417e3e2f1333481795a089bef2e24bea&role.id=70c34ea493484322911dab62b4a56e75 => generated 608 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 21:59:40.288377 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5f370e-4008-4a2b-8ff8-38d11507d3c9 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:40.407908 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c5f370e-4008-4a2b-8ff8-38d11507d3c9 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:40.424887 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-297629a0-176e-4623-9ee8-e0447f450afc None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:40.451895 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee577958-cd0a-48a0-a2a3-71b401548b6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:40.453654 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee577958-cd0a-48a0-a2a3-71b401548b6b None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:40.454282 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee577958-cd0a-48a0-a2a3-71b401548b6b None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:40.454805 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee577958-cd0a-48a0-a2a3-71b401548b6b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:40.486545 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:59:40.523285 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17d66f8e-df8f-4af3-bc9f-d2a5ab2add4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:40.524141 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d66f8e-df8f-4af3-bc9f-d2a5ab2add4b None admin] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:40.524463 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d66f8e-df8f-4af3-bc9f-d2a5ab2add4b None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:40.524740 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17d66f8e-df8f-4af3-bc9f-d2a5ab2add4b None admin] PATH_INFO: `/v3/users` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:40.525463 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d66f8e-df8f-4af3-bc9f-d2a5ab2add4b None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:40.527169 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17d66f8e-df8f-4af3-bc9f-d2a5ab2add4b None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:40.540802 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee577958-cd0a-48a0-a2a3-71b401548b6b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:40.542787 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ee577958-cd0a-48a0-a2a3-71b401548b6b None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:40.575312 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 346/703] 10.222.0.52 () {60 vars in 1132 bytes} [Wed Jul 13 21:59:40 2022] POST /identity/v3/users => generated 254 bytes in 330 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 21:59:40.600601 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de35bc9d-df11-4124-911e-b1faf9d5ebe3 None None] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:40.600889 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de35bc9d-df11-4124-911e-b1faf9d5ebe3 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:40.601282 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de35bc9d-df11-4124-911e-b1faf9d5ebe3 None None] PATH_INFO: `/` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:40.603283 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 347/704] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jul 13 21:59:40 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 13 21:59:40.610174 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecf5d3c-9659-4112-b6fc-cdf692e6830c None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:40.610524 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecf5d3c-9659-4112-b6fc-cdf692e6830c None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:40.610761 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ecf5d3c-9659-4112-b6fc-cdf692e6830c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:40.634136 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-2ecf5d3c-9659-4112-b6fc-cdf692e6830c None None] MFA Rules not processed for user `cc99c93da5074cf5a48bc2aa9a92c5e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:59:40.635514 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-2ecf5d3c-9659-4112-b6fc-cdf692e6830c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:59:40.683779 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 358/705] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Jul 13 21:59:40 2022] GET /identity/v3/auth/tokens => generated 799 bytes in 395 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 13 21:59:40.690151 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee577958-cd0a-48a0-a2a3-71b401548b6b None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:40.693684 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:59:40.725454 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 348/706] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jul 13 21:59:40 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 117 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 13 21:59:40.748705 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ecf5d3c-9659-4112-b6fc-cdf692e6830c None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:40.760752 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:59:40.814001 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee577958-cd0a-48a0-a2a3-71b401548b6b None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:40.862888 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd3b33e-e3cd-436d-a8f7-7a742599a66a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:40.864596 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd3b33e-e3cd-436d-a8f7-7a742599a66a None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:40.865105 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd3b33e-e3cd-436d-a8f7-7a742599a66a None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:40.865528 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cd3b33e-e3cd-436d-a8f7-7a742599a66a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:40.875601 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ecf5d3c-9659-4112-b6fc-cdf692e6830c None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:40.882712 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd3b33e-e3cd-436d-a8f7-7a742599a66a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:40.884250 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cd3b33e-e3cd-436d-a8f7-7a742599a66a None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:40.913928 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application [None req-1cd3b33e-e3cd-436d-a8f7-7a742599a66a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 21:59:40.913928 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 21:59:40.913928 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 21:59:40.913928 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 21:59:40.913928 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 21:59:40.913928 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 21:59:40.913928 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 21:59:40.913928 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 21:59:40.913928 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 21:59:40.913928 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 21:59:40.913928 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 21:59:40.913928 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 21:59:40.913928 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 21:59:40.913928 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 21:59:40.913928 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 21:59:40.914931 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 21:59:40.914931 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 21:59:40.914931 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 21:59:40.914931 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 21:59:40.914931 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 21:59:40.914931 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 21:59:40.914931 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 21:59:40.914931 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 21:59:40.914931 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application with Lock( Jul 13 21:59:40.914931 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 21:59:40.914931 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._enter() Jul 13 21:59:40.914931 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 21:59:40.914931 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 21:59:40.914931 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 21:59:40.914931 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.creator() Jul 13 21:59:40.914931 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 21:59:40.914931 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application created_value = creator( Jul 13 21:59:40.915876 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 21:59:40.915876 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 21:59:40.915876 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 21:59:40.915876 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Jul 13 21:59:40.915876 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 359/707] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Jul 13 21:59:40 2022] GET /identity/v3/domains/Default => generated 87 bytes in 226 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 13 21:59:40.921054 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd3b33e-e3cd-436d-a8f7-7a742599a66a None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:40.924373 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7e6632-b8aa-4c50-8cc9-24dc3e34eb53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:40.925457 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7e6632-b8aa-4c50-8cc9-24dc3e34eb53 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:40.925739 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7e6632-b8aa-4c50-8cc9-24dc3e34eb53 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:40.927029 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab7e6632-b8aa-4c50-8cc9-24dc3e34eb53 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:40.927714 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7e6632-b8aa-4c50-8cc9-24dc3e34eb53 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:40.929162 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab7e6632-b8aa-4c50-8cc9-24dc3e34eb53 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:40.947460 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application [None req-ab7e6632-b8aa-4c50-8cc9-24dc3e34eb53 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 13 21:59:40.947460 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 21:59:40.947460 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 13 21:59:40.947460 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 13 21:59:40.947460 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 13 21:59:40.947460 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 13 21:59:40.947460 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 13 21:59:40.947460 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 13 21:59:40.947460 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 13 21:59:40.947460 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Jul 13 21:59:40.947460 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 13 21:59:40.947460 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Jul 13 21:59:40.947460 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 21:59:40.947460 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 21:59:40.947460 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 21:59:40.947460 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 21:59:40.948517 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 21:59:40.948517 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 21:59:40.948517 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 21:59:40.948517 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 21:59:40.948517 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 21:59:40.948517 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 21:59:40.948517 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 21:59:40.948517 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 13 21:59:40.948517 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 13 21:59:40.948517 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 13 21:59:40.948517 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 13 21:59:40.948517 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 21:59:40.948517 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 21:59:40.948517 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 21:59:40.948517 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 21:59:40.948517 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 21:59:40.950644 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 21:59:40.950644 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 21:59:40.950644 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application with Lock( Jul 13 21:59:40.950644 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 21:59:40.950644 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._enter() Jul 13 21:59:40.950644 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 21:59:40.950644 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 21:59:40.950644 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 21:59:40.950644 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.creator() Jul 13 21:59:40.950644 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 21:59:40.950644 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application created_value = creator( Jul 13 21:59:40.950644 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 13 21:59:40.950644 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 13 21:59:40.950644 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 13 21:59:40.950644 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Jul 13 21:59:40.950644 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 349/708] 10.222.0.52 () {56 vars in 1129 bytes} [Wed Jul 13 21:59:40 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 21:59:40.955563 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7e6632-b8aa-4c50-8cc9-24dc3e34eb53 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:41.006226 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:59:41.079033 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cd3b33e-e3cd-436d-a8f7-7a742599a66a None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:41.089431 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:59:41.124279 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab7e6632-b8aa-4c50-8cc9-24dc3e34eb53 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:41.140119 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c9cb05-cf47-4297-a2ca-ab7af7e25ae9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:41.140849 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c9cb05-cf47-4297-a2ca-ab7af7e25ae9 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:41.141126 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c9cb05-cf47-4297-a2ca-ab7af7e25ae9 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:41.141398 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12c9cb05-cf47-4297-a2ca-ab7af7e25ae9 None admin] PATH_INFO: `/v3/domains` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:41.142181 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c9cb05-cf47-4297-a2ca-ab7af7e25ae9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 21:59:41.142506 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c9cb05-cf47-4297-a2ca-ab7af7e25ae9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:41.143904 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12c9cb05-cf47-4297-a2ca-ab7af7e25ae9 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:41.158050 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 360/709] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Jul 13 21:59:40 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 237 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 21:59:41.163877 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c9cb05-cf47-4297-a2ca-ab7af7e25ae9 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:41.198243 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd297630-bd13-468f-9186-1c9a1597320d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:41.198859 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd297630-bd13-468f-9186-1c9a1597320d None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:41.199490 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd297630-bd13-468f-9186-1c9a1597320d None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:41.199759 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd297630-bd13-468f-9186-1c9a1597320d None admin] PATH_INFO: `/v3/services` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:41.200481 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd297630-bd13-468f-9186-1c9a1597320d None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 21:59:41.200787 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd297630-bd13-468f-9186-1c9a1597320d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:41.202168 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd297630-bd13-468f-9186-1c9a1597320d None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:41.212109 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 350/710] 10.222.0.52 () {56 vars in 1124 bytes} [Wed Jul 13 21:59:40 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 256 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 21:59:41.217888 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd297630-bd13-468f-9186-1c9a1597320d None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:41.323941 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12c9cb05-cf47-4297-a2ca-ab7af7e25ae9 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:41.350307 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:59:41.354865 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd297630-bd13-468f-9186-1c9a1597320d None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:41.392785 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e77935-6e3d-484d-865d-dd3b99508ff5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:41.393465 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e77935-6e3d-484d-865d-dd3b99508ff5 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:41.393743 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e77935-6e3d-484d-865d-dd3b99508ff5 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:41.394015 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e77935-6e3d-484d-865d-dd3b99508ff5 None admin] PATH_INFO: `/v3/services` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:41.394755 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e77935-6e3d-484d-865d-dd3b99508ff5 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 21:59:41.395060 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e77935-6e3d-484d-865d-dd3b99508ff5 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:41.395429 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257e425e-60b1-42d4-93bd-419939665eee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:41.396429 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4e77935-6e3d-484d-865d-dd3b99508ff5 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:41.396668 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257e425e-60b1-42d4-93bd-419939665eee None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:41.397147 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257e425e-60b1-42d4-93bd-419939665eee None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:41.397625 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-257e425e-60b1-42d4-93bd-419939665eee None admin] PATH_INFO: `/v3/projects/service` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:41.403492 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 351/711] 10.222.0.52 () {56 vars in 1124 bytes} [Wed Jul 13 21:59:41 2022] GET /identity/v3/services?type=block-storage => generated 130 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 21:59:41.405267 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a779370-2550-4112-aa5e-02e0f229382b None None] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:41.405477 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-257e425e-60b1-42d4-93bd-419939665eee None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:41.405887 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a779370-2550-4112-aa5e-02e0f229382b None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:41.406299 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a779370-2550-4112-aa5e-02e0f229382b None None] PATH_INFO: `/` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:41.407257 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-257e425e-60b1-42d4-93bd-419939665eee None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:41.408184 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 352/712] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jul 13 21:59:41 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 21:59:41.416158 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd276ff-1447-4286-8be5-368da2647289 None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:41.416519 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd276ff-1447-4286-8be5-368da2647289 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:41.416888 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cd276ff-1447-4286-8be5-368da2647289 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:41.426976 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application [None req-257e425e-60b1-42d4-93bd-419939665eee None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 13 21:59:41.426976 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 21:59:41.426976 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 21:59:41.426976 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 21:59:41.426976 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 21:59:41.426976 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 21:59:41.426976 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 21:59:41.426976 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 21:59:41.426976 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 21:59:41.426976 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 21:59:41.426976 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 21:59:41.426976 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 21:59:41.426976 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 13 21:59:41.426976 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 13 21:59:41.426976 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 13 21:59:41.427992 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 13 21:59:41.427992 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 21:59:41.427992 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 21:59:41.427992 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 21:59:41.427992 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 21:59:41.427992 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 21:59:41.427992 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 21:59:41.427992 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 21:59:41.427992 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application with Lock( Jul 13 21:59:41.427992 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 21:59:41.427992 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._enter() Jul 13 21:59:41.427992 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 21:59:41.427992 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 21:59:41.427992 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 21:59:41.427992 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.creator() Jul 13 21:59:41.427992 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 21:59:41.427992 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application created_value = creator( Jul 13 21:59:41.428939 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 13 21:59:41.428939 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 13 21:59:41.428939 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 13 21:59:41.428939 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 13 21:59:41.428939 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 13 21:59:41.428939 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 13 21:59:41.428939 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 13 21:59:41.428939 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Jul 13 21:59:41.435506 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 361/713] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Jul 13 21:59:41 2022] GET /identity/v3/projects/service => generated 88 bytes in 267 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 21:59:41.440190 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257e425e-60b1-42d4-93bd-419939665eee None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:41.461694 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-8cd276ff-1447-4286-8be5-368da2647289 None None] MFA Rules not processed for user `cc99c93da5074cf5a48bc2aa9a92c5e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:59:41.462953 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-8cd276ff-1447-4286-8be5-368da2647289 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:59:41.510967 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:59:41.581192 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 353/714] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jul 13 21:59:41 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 13 21:59:41.601092 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-257e425e-60b1-42d4-93bd-419939665eee None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:41.603827 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd276ff-1447-4286-8be5-368da2647289 None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:41.639895 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2a0508-d625-4053-a589-afcf013e9d36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:41.639895 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2a0508-d625-4053-a589-afcf013e9d36 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:41.640232 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2a0508-d625-4053-a589-afcf013e9d36 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:41.640466 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de2a0508-d625-4053-a589-afcf013e9d36 None admin] PATH_INFO: `/v3/projects` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:41.641276 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de2a0508-d625-4053-a589-afcf013e9d36 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 21:59:41.641593 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de2a0508-d625-4053-a589-afcf013e9d36 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:41.643090 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de2a0508-d625-4053-a589-afcf013e9d36 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:41.656624 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 362/715] 10.222.0.52 () {56 vars in 1148 bytes} [Wed Jul 13 21:59:41 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 217 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 21:59:41.663355 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2a0508-d625-4053-a589-afcf013e9d36 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:41.702145 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:59:41.712264 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cd276ff-1447-4286-8be5-368da2647289 None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:41.747807 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e10d1d-18df-4993-a9b8-92d0e9592571 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:41.748615 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e10d1d-18df-4993-a9b8-92d0e9592571 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:41.748891 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e10d1d-18df-4993-a9b8-92d0e9592571 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:41.749159 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3e10d1d-18df-4993-a9b8-92d0e9592571 None admin] PATH_INFO: `/v3/services/network` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:41.749817 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e10d1d-18df-4993-a9b8-92d0e9592571 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:41.751226 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3e10d1d-18df-4993-a9b8-92d0e9592571 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:41.762971 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:59:41.765598 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application [None req-b3e10d1d-18df-4993-a9b8-92d0e9592571 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 13 21:59:41.765598 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 21:59:41.765598 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 13 21:59:41.765598 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 13 21:59:41.765598 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 13 21:59:41.765598 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 13 21:59:41.765598 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 13 21:59:41.765598 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 13 21:59:41.765598 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 13 21:59:41.765598 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Jul 13 21:59:41.765598 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 13 21:59:41.765598 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Jul 13 21:59:41.765598 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 21:59:41.765598 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 21:59:41.765598 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 21:59:41.765598 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 21:59:41.765598 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 21:59:41.766890 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 21:59:41.766890 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 21:59:41.766890 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 21:59:41.766890 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 21:59:41.766890 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 21:59:41.766890 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 21:59:41.766890 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 13 21:59:41.766890 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 13 21:59:41.766890 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 13 21:59:41.766890 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 13 21:59:41.766890 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 21:59:41.766890 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 21:59:41.766890 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 21:59:41.766890 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 21:59:41.766890 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 21:59:41.766890 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 21:59:41.767928 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 21:59:41.767928 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application with Lock( Jul 13 21:59:41.767928 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 21:59:41.767928 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._enter() Jul 13 21:59:41.767928 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 21:59:41.767928 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 21:59:41.767928 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 21:59:41.767928 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.creator() Jul 13 21:59:41.767928 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 21:59:41.767928 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application created_value = creator( Jul 13 21:59:41.767928 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 13 21:59:41.767928 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 13 21:59:41.767928 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 13 21:59:41.767928 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Jul 13 21:59:41.770317 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 354/716] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Jul 13 21:59:41 2022] GET /identity/v3/services/network => generated 88 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 21:59:41.776463 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de2a0508-d625-4053-a589-afcf013e9d36 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:41.776724 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e10d1d-18df-4993-a9b8-92d0e9592571 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:41.813578 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19f5bf6-a121-4626-b084-240102144548 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:41.814249 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19f5bf6-a121-4626-b084-240102144548 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:41.814556 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19f5bf6-a121-4626-b084-240102144548 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:41.814989 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19f5bf6-a121-4626-b084-240102144548 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:41.816476 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19f5bf6-a121-4626-b084-240102144548 None admin] RBAC: Adding query filter params (role.id=44efba2579f342df8a0789c59d163ec3, scope.project.id=417e3e2f1333481795a089bef2e24bea, user.id=f84281091687464990ffbca590b9343b) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 21:59:41.816476 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19f5bf6-a121-4626-b084-240102144548 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:41.817772 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19f5bf6-a121-4626-b084-240102144548 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:41.824718 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 363/717] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Jul 13 21:59:41 2022] GET /identity/v3/role_assignments?user.id=f84281091687464990ffbca590b9343b&scope.project.id=417e3e2f1333481795a089bef2e24bea&role.id=44efba2579f342df8a0789c59d163ec3 => generated 259 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 21:59:41.899784 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3e10d1d-18df-4993-a9b8-92d0e9592571 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:41.945558 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6844e1a8-94e0-4a03-91c5-1f273332e3bb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:41.946663 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6844e1a8-94e0-4a03-91c5-1f273332e3bb None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:41.947506 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6844e1a8-94e0-4a03-91c5-1f273332e3bb None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:41.947948 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6844e1a8-94e0-4a03-91c5-1f273332e3bb None admin] PATH_INFO: `/v3/services` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:41.948837 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6844e1a8-94e0-4a03-91c5-1f273332e3bb None admin] RBAC: Adding query filter params (name=network) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 21:59:41.949681 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6844e1a8-94e0-4a03-91c5-1f273332e3bb None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:41.951267 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6844e1a8-94e0-4a03-91c5-1f273332e3bb None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:41.956716 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 355/718] 10.222.0.52 () {56 vars in 1112 bytes} [Wed Jul 13 21:59:41 2022] GET /identity/v3/services?name=network => generated 124 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 21:59:41.963867 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19f5bf6-a121-4626-b084-240102144548 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:42.098319 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19f5bf6-a121-4626-b084-240102144548 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:42.147172 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e64106db-6c10-4293-93ac-d1cf49247b88 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:42.148154 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64106db-6c10-4293-93ac-d1cf49247b88 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:42.148607 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64106db-6c10-4293-93ac-d1cf49247b88 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:42.149051 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e64106db-6c10-4293-93ac-d1cf49247b88 None admin] PATH_INFO: `/v3/services` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:42.149994 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64106db-6c10-4293-93ac-d1cf49247b88 None admin] RBAC: Adding query filter params (type=network) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 21:59:42.150606 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64106db-6c10-4293-93ac-d1cf49247b88 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:42.152151 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e64106db-6c10-4293-93ac-d1cf49247b88 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:42.163627 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 364/719] 10.222.0.52 () {56 vars in 1112 bytes} [Wed Jul 13 21:59:41 2022] GET /identity/v3/services?type=network => generated 124 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 21:59:43.122696 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6566fec5-433a-4ce0-a182-8bba83678004 None None] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:43.123272 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6566fec5-433a-4ce0-a182-8bba83678004 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:43.123272 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6566fec5-433a-4ce0-a182-8bba83678004 None None] PATH_INFO: `/` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:43.124906 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 356/720] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jul 13 21:59:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 21:59:43.130068 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e047a98-f28f-4649-b681-437d86dd5244 None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:43.130478 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e047a98-f28f-4649-b681-437d86dd5244 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:43.130702 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e047a98-f28f-4649-b681-437d86dd5244 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:43.139383 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6f31b8-402d-4b17-a2f6-5c1f339b26f8 None None] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:43.139669 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6f31b8-402d-4b17-a2f6-5c1f339b26f8 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:43.139947 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd6f31b8-402d-4b17-a2f6-5c1f339b26f8 None None] PATH_INFO: `/` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:43.141543 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 357/721] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jul 13 21:59:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 21:59:43.146893 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45b415a-3633-4648-bca8-550c0b5ade8f None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:43.147572 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45b415a-3633-4648-bca8-550c0b5ade8f None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:43.147855 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f45b415a-3633-4648-bca8-550c0b5ade8f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:43.167637 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-4e047a98-f28f-4649-b681-437d86dd5244 None None] MFA Rules not processed for user `cc99c93da5074cf5a48bc2aa9a92c5e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:59:43.168848 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-4e047a98-f28f-4649-b681-437d86dd5244 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:59:43.175521 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-f45b415a-3633-4648-bca8-550c0b5ade8f None None] MFA Rules not processed for user `cc99c93da5074cf5a48bc2aa9a92c5e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:59:43.176680 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-f45b415a-3633-4648-bca8-550c0b5ade8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:59:43.189012 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:59:43.205085 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:59:43.257324 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 365/722] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jul 13 21:59:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 13 21:59:43.281361 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e047a98-f28f-4649-b681-437d86dd5244 None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:43.333200 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 358/723] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jul 13 21:59:43 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 186 msecs (HTTP/1.1 201) 6 headers in 363 bytes (2 switches on core 0) Jul 13 21:59:43.357040 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45b415a-3633-4648-bca8-550c0b5ade8f None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:43.484884 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e047a98-f28f-4649-b681-437d86dd5244 None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:43.525646 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f51697a-20ac-4fdf-8a54-818fe5477bd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:43.527015 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f51697a-20ac-4fdf-8a54-818fe5477bd7 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:43.527296 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f51697a-20ac-4fdf-8a54-818fe5477bd7 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:43.527568 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f51697a-20ac-4fdf-8a54-818fe5477bd7 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:43.544286 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f51697a-20ac-4fdf-8a54-818fe5477bd7 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 21:59:43.558512 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application [None req-6f51697a-20ac-4fdf-8a54-818fe5477bd7 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 13 21:59:43.558512 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 21:59:43.558512 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 21:59:43.558512 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 21:59:43.558512 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 21:59:43.558512 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 21:59:43.558512 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 21:59:43.558512 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 21:59:43.558512 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 21:59:43.558512 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 21:59:43.558512 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 21:59:43.558512 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 21:59:43.558512 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 21:59:43.558512 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 21:59:43.558512 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 21:59:43.558512 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 21:59:43.559510 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 21:59:43.559510 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 21:59:43.559510 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 21:59:43.559510 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 21:59:43.559510 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 21:59:43.559510 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 21:59:43.559510 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 21:59:43.559510 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 21:59:43.559510 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 21:59:43.559510 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 21:59:43.559510 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 21:59:43.559510 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application with Lock( Jul 13 21:59:43.559510 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 21:59:43.559510 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._enter() Jul 13 21:59:43.559510 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 21:59:43.559510 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 21:59:43.560352 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 21:59:43.560352 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.creator() Jul 13 21:59:43.560352 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 21:59:43.560352 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application created_value = creator( Jul 13 21:59:43.560352 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 21:59:43.560352 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 21:59:43.560352 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 21:59:43.560352 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 21:59:43.560352 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 21:59:43.560352 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 21:59:43.560352 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 13 21:59:43.560352 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Jul 13 21:59:43.560352 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 366/724] 10.222.0.52 () {56 vars in 1102 bytes} [Wed Jul 13 21:59:43 2022] GET /identity/v3/roles/service => generated 85 bytes in 273 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 13 21:59:43.561059 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f45b415a-3633-4648-bca8-550c0b5ade8f None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:43.563173 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f51697a-20ac-4fdf-8a54-818fe5477bd7 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:43.577762 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:59:43.600912 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf532239-344a-4104-86b7-780f19f56964 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:43.602022 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf532239-344a-4104-86b7-780f19f56964 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:43.602305 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf532239-344a-4104-86b7-780f19f56964 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:43.602680 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf532239-344a-4104-86b7-780f19f56964 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:43.615150 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf532239-344a-4104-86b7-780f19f56964 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 21:59:43.623833 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application [None req-cf532239-344a-4104-86b7-780f19f56964 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 13 21:59:43.623833 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 21:59:43.623833 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 21:59:43.623833 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 21:59:43.623833 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 21:59:43.623833 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 21:59:43.623833 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 21:59:43.623833 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 21:59:43.623833 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 21:59:43.623833 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 21:59:43.623833 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 21:59:43.623833 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 21:59:43.623833 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 21:59:43.623833 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 21:59:43.623833 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 21:59:43.623833 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 21:59:43.624818 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 21:59:43.624818 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 21:59:43.624818 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 21:59:43.624818 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 21:59:43.624818 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 21:59:43.624818 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 21:59:43.624818 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 21:59:43.624818 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 21:59:43.624818 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 21:59:43.624818 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 21:59:43.624818 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 21:59:43.624818 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application with Lock( Jul 13 21:59:43.624818 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 21:59:43.624818 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._enter() Jul 13 21:59:43.624818 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 21:59:43.624818 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 21:59:43.625762 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 21:59:43.625762 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.creator() Jul 13 21:59:43.625762 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 21:59:43.625762 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application created_value = creator( Jul 13 21:59:43.625762 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 21:59:43.625762 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 21:59:43.625762 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 21:59:43.625762 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 21:59:43.625762 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 21:59:43.625762 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 21:59:43.625762 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 13 21:59:43.625762 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Jul 13 21:59:43.625762 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 359/725] 10.222.0.52 () {56 vars in 1096 bytes} [Wed Jul 13 21:59:43 2022] GET /identity/v3/roles/admin => generated 83 bytes in 269 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 13 21:59:43.635701 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf532239-344a-4104-86b7-780f19f56964 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:43.651252 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:59:43.747072 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf532239-344a-4104-86b7-780f19f56964 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:43.783125 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a82bff-ad68-4d7b-b9b8-a92c970233b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:43.783946 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a82bff-ad68-4d7b-b9b8-a92c970233b5 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:43.784597 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a82bff-ad68-4d7b-b9b8-a92c970233b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:43.784972 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a82bff-ad68-4d7b-b9b8-a92c970233b5 None admin] PATH_INFO: `/v3/roles` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:43.785973 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a82bff-ad68-4d7b-b9b8-a92c970233b5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 21:59:43.786389 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a82bff-ad68-4d7b-b9b8-a92c970233b5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:43.788191 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a82bff-ad68-4d7b-b9b8-a92c970233b5 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:43.796413 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f51697a-20ac-4fdf-8a54-818fe5477bd7 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:43.797485 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 360/726] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Jul 13 21:59:43 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 21:59:43.802638 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a82bff-ad68-4d7b-b9b8-a92c970233b5 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:43.875744 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55a338b-42c7-4ac1-99a4-c87d412fba18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:43.876734 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55a338b-42c7-4ac1-99a4-c87d412fba18 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:43.877229 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55a338b-42c7-4ac1-99a4-c87d412fba18 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:43.877703 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f55a338b-42c7-4ac1-99a4-c87d412fba18 None admin] PATH_INFO: `/v3/roles` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:43.878709 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55a338b-42c7-4ac1-99a4-c87d412fba18 None admin] RBAC: Adding query filter params (name=service) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 21:59:43.879238 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55a338b-42c7-4ac1-99a4-c87d412fba18 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:43.880817 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f55a338b-42c7-4ac1-99a4-c87d412fba18 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:43.897412 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 367/727] 10.222.0.52 () {56 vars in 1103 bytes} [Wed Jul 13 21:59:43 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 335 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 21:59:43.902518 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:59:43.904132 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55a338b-42c7-4ac1-99a4-c87d412fba18 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:43.916348 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a82bff-ad68-4d7b-b9b8-a92c970233b5 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:43.969795 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6163d368-b3a9-4c6b-be40-e8200070bf85 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:43.970639 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6163d368-b3a9-4c6b-be40-e8200070bf85 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:43.970920 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6163d368-b3a9-4c6b-be40-e8200070bf85 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:43.971194 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6163d368-b3a9-4c6b-be40-e8200070bf85 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:43.985266 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6163d368-b3a9-4c6b-be40-e8200070bf85 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:43.986839 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6163d368-b3a9-4c6b-be40-e8200070bf85 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:44.005762 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application [None req-6163d368-b3a9-4c6b-be40-e8200070bf85 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 21:59:44.005762 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 21:59:44.005762 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 21:59:44.005762 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 21:59:44.005762 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 21:59:44.005762 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 21:59:44.005762 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 21:59:44.005762 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 21:59:44.005762 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 21:59:44.005762 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 21:59:44.005762 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 21:59:44.005762 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 21:59:44.005762 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 21:59:44.005762 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 21:59:44.005762 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 21:59:44.006823 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 21:59:44.006823 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 21:59:44.006823 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 21:59:44.006823 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 21:59:44.006823 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 21:59:44.006823 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 21:59:44.006823 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 21:59:44.006823 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 21:59:44.006823 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application with Lock( Jul 13 21:59:44.006823 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 21:59:44.006823 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._enter() Jul 13 21:59:44.006823 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 21:59:44.006823 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 21:59:44.006823 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 21:59:44.006823 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.creator() Jul 13 21:59:44.006823 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 21:59:44.006823 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application created_value = creator( Jul 13 21:59:44.007733 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 21:59:44.007733 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 21:59:44.007733 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 21:59:44.007733 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Jul 13 21:59:44.007733 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 361/728] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Jul 13 21:59:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 21:59:44.008225 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999705fa-e6bd-49c5-b3f5-b947478189ea None None] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:44.008446 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999705fa-e6bd-49c5-b3f5-b947478189ea None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:44.011950 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999705fa-e6bd-49c5-b3f5-b947478189ea None None] PATH_INFO: `/` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:44.014831 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 362/729] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jul 13 21:59:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 21:59:44.015637 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999705fa-e6bd-49c5-b3f5-b947478189ea None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:44.020651 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:59:44.065540 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f55a338b-42c7-4ac1-99a4-c87d412fba18 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:44.121391 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df21320a-07cb-42d0-9fa7-2006c84e326d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:44.123204 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df21320a-07cb-42d0-9fa7-2006c84e326d None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:44.123637 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df21320a-07cb-42d0-9fa7-2006c84e326d None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:44.124340 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df21320a-07cb-42d0-9fa7-2006c84e326d None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:44.133361 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999705fa-e6bd-49c5-b3f5-b947478189ea None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:44.137767 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df21320a-07cb-42d0-9fa7-2006c84e326d None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:44.139282 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df21320a-07cb-42d0-9fa7-2006c84e326d None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:44.157186 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application [None req-df21320a-07cb-42d0-9fa7-2006c84e326d None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 21:59:44.157186 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 21:59:44.157186 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 21:59:44.157186 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 21:59:44.157186 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 21:59:44.157186 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 21:59:44.157186 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 21:59:44.157186 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 21:59:44.157186 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 21:59:44.157186 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 21:59:44.157186 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 21:59:44.157186 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 21:59:44.157186 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 21:59:44.157186 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 21:59:44.157186 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 21:59:44.158332 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 21:59:44.158332 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 21:59:44.158332 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 21:59:44.158332 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 21:59:44.158332 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 21:59:44.158332 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 21:59:44.158332 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 21:59:44.158332 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 21:59:44.158332 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application with Lock( Jul 13 21:59:44.158332 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 21:59:44.158332 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._enter() Jul 13 21:59:44.158332 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 21:59:44.158332 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 21:59:44.158332 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 21:59:44.158332 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.creator() Jul 13 21:59:44.158332 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 21:59:44.158332 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application created_value = creator( Jul 13 21:59:44.159465 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 21:59:44.159465 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 21:59:44.159465 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 21:59:44.159465 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Jul 13 21:59:44.163263 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:59:44.163393 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 368/730] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Jul 13 21:59:43 2022] GET /identity/v3/domains/Default => generated 87 bytes in 259 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 13 21:59:44.165769 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e9275c-22a8-47ce-beec-b328ab7ee6f4 None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:44.166245 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e9275c-22a8-47ce-beec-b328ab7ee6f4 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:44.166728 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93e9275c-22a8-47ce-beec-b328ab7ee6f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:44.178379 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af74dd9d-6fdc-47e8-86fd-be4687cf7faf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:44.179411 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af74dd9d-6fdc-47e8-86fd-be4687cf7faf None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:44.179859 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af74dd9d-6fdc-47e8-86fd-be4687cf7faf None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:44.180280 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af74dd9d-6fdc-47e8-86fd-be4687cf7faf None admin] PATH_INFO: `/v3/domains` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:44.181442 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af74dd9d-6fdc-47e8-86fd-be4687cf7faf None admin] RBAC: Adding query filter params (name=Default) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 21:59:44.181905 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af74dd9d-6fdc-47e8-86fd-be4687cf7faf None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:44.183629 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af74dd9d-6fdc-47e8-86fd-be4687cf7faf None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:44.188382 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-93e9275c-22a8-47ce-beec-b328ab7ee6f4 None None] MFA Rules not processed for user `cc99c93da5074cf5a48bc2aa9a92c5e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:59:44.189638 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-93e9275c-22a8-47ce-beec-b328ab7ee6f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:59:44.201969 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 363/731] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Jul 13 21:59:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 182 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 21:59:44.201969 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af74dd9d-6fdc-47e8-86fd-be4687cf7faf None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:44.291329 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 369/732] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jul 13 21:59:44 2022] POST /identity/v3/auth/tokens => generated 799 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 13 21:59:44.292270 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e9275c-22a8-47ce-beec-b328ab7ee6f4 None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:44.320019 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af74dd9d-6fdc-47e8-86fd-be4687cf7faf None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:44.365140 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:59:44.367475 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786c208f-b79f-4d2c-be48-60bd13e4446e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:44.369706 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786c208f-b79f-4d2c-be48-60bd13e4446e None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:44.370225 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786c208f-b79f-4d2c-be48-60bd13e4446e None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:44.370984 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-786c208f-b79f-4d2c-be48-60bd13e4446e None admin] PATH_INFO: `/v3/domains` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:44.372015 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786c208f-b79f-4d2c-be48-60bd13e4446e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 21:59:44.372634 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786c208f-b79f-4d2c-be48-60bd13e4446e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:44.375168 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-786c208f-b79f-4d2c-be48-60bd13e4446e None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:44.394274 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 364/733] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Jul 13 21:59:44 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 21:59:44.395248 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786c208f-b79f-4d2c-be48-60bd13e4446e None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:44.430674 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93e9275c-22a8-47ce-beec-b328ab7ee6f4 None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:44.446001 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:59:44.468297 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e6e8fe-db44-40aa-abbf-d0dddcf65b5e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:44.469407 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e6e8fe-db44-40aa-abbf-d0dddcf65b5e None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:44.469873 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e6e8fe-db44-40aa-abbf-d0dddcf65b5e None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:44.470141 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98e6e8fe-db44-40aa-abbf-d0dddcf65b5e None admin] PATH_INFO: `/v3/users/swift` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:44.477923 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e6e8fe-db44-40aa-abbf-d0dddcf65b5e None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:44.479600 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98e6e8fe-db44-40aa-abbf-d0dddcf65b5e None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:44.494737 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application [None req-98e6e8fe-db44-40aa-abbf-d0dddcf65b5e None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 13 21:59:44.494737 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 21:59:44.494737 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 21:59:44.494737 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 21:59:44.494737 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 21:59:44.494737 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 21:59:44.494737 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 21:59:44.494737 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 21:59:44.494737 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 21:59:44.494737 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 21:59:44.494737 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 21:59:44.494737 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 21:59:44.494737 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 13 21:59:44.494737 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 13 21:59:44.494737 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 13 21:59:44.494737 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 13 21:59:44.495724 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 21:59:44.495724 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 21:59:44.495724 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 13 21:59:44.495724 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 21:59:44.495724 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 13 21:59:44.495724 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 21:59:44.495724 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 21:59:44.495724 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 21:59:44.495724 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 21:59:44.495724 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 21:59:44.495724 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 21:59:44.495724 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application with Lock( Jul 13 21:59:44.495724 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 21:59:44.495724 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._enter() Jul 13 21:59:44.495724 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 21:59:44.495724 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 21:59:44.495724 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 21:59:44.495724 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.creator() Jul 13 21:59:44.496657 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 21:59:44.496657 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application created_value = creator( Jul 13 21:59:44.496657 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 13 21:59:44.496657 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 13 21:59:44.496657 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 13 21:59:44.496657 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 13 21:59:44.496657 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 13 21:59:44.496657 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 13 21:59:44.496657 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 13 21:59:44.496657 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Jul 13 21:59:44.496657 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 370/734] 10.222.0.52 () {56 vars in 1096 bytes} [Wed Jul 13 21:59:44 2022] GET /identity/v3/users/swift => generated 83 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 21:59:44.497262 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e6e8fe-db44-40aa-abbf-d0dddcf65b5e None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:44.506930 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-786c208f-b79f-4d2c-be48-60bd13e4446e None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:44.551246 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0e67315-9fab-4a60-ad3a-c075e0bb2278 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:44.552802 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e67315-9fab-4a60-ad3a-c075e0bb2278 None admin] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:44.553079 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e67315-9fab-4a60-ad3a-c075e0bb2278 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:44.553390 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0e67315-9fab-4a60-ad3a-c075e0bb2278 None admin] PATH_INFO: `/v3/services` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:44.554087 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e67315-9fab-4a60-ad3a-c075e0bb2278 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:44.555489 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0e67315-9fab-4a60-ad3a-c075e0bb2278 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:44.587331 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 365/735] 10.222.0.52 () {60 vars in 1141 bytes} [Wed Jul 13 21:59:44 2022] POST /identity/v3/services => generated 251 bytes in 192 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 21:59:44.589453 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871d6128-8f2e-4a85-874d-a91776559be4 None None] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:44.589745 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871d6128-8f2e-4a85-874d-a91776559be4 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:44.590023 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-871d6128-8f2e-4a85-874d-a91776559be4 None None] PATH_INFO: `/` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:44.594966 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 366/736] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jul 13 21:59:44 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 21:59:44.596234 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871d6128-8f2e-4a85-874d-a91776559be4 None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:44.613097 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98e6e8fe-db44-40aa-abbf-d0dddcf65b5e None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:44.627727 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:59:44.665268 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fadfa41-48a9-411e-8863-e909886595f7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:44.666904 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fadfa41-48a9-411e-8863-e909886595f7 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:44.667450 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fadfa41-48a9-411e-8863-e909886595f7 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:44.667953 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fadfa41-48a9-411e-8863-e909886595f7 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:44.676657 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fadfa41-48a9-411e-8863-e909886595f7 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:44.678602 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fadfa41-48a9-411e-8863-e909886595f7 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:44.695567 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application [None req-2fadfa41-48a9-411e-8863-e909886595f7 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 13 21:59:44.695567 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 21:59:44.695567 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 21:59:44.695567 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 21:59:44.695567 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 21:59:44.695567 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 21:59:44.695567 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 21:59:44.695567 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 21:59:44.695567 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 21:59:44.695567 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 21:59:44.695567 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 21:59:44.695567 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 21:59:44.695567 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 13 21:59:44.695567 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 13 21:59:44.695567 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 13 21:59:44.695567 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 13 21:59:44.696666 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 21:59:44.696666 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 21:59:44.696666 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 13 21:59:44.696666 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 21:59:44.696666 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 13 21:59:44.696666 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 21:59:44.696666 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 21:59:44.696666 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 21:59:44.696666 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 21:59:44.696666 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 21:59:44.696666 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 21:59:44.696666 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application with Lock( Jul 13 21:59:44.696666 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 21:59:44.696666 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._enter() Jul 13 21:59:44.696666 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 21:59:44.696666 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 21:59:44.696666 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 21:59:44.696666 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.creator() Jul 13 21:59:44.697618 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 21:59:44.697618 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application created_value = creator( Jul 13 21:59:44.697618 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 13 21:59:44.697618 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 13 21:59:44.697618 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 13 21:59:44.697618 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 13 21:59:44.697618 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 13 21:59:44.697618 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 13 21:59:44.697618 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 13 21:59:44.697618 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Jul 13 21:59:44.700876 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 371/737] 10.222.0.52 () {56 vars in 1117 bytes} [Wed Jul 13 21:59:44 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 204 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 21:59:44.702681 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8defbc1-dae0-4346-acb3-595d9c6842c3 None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:44.703191 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8defbc1-dae0-4346-acb3-595d9c6842c3 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:44.703676 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e8defbc1-dae0-4346-acb3-595d9c6842c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:44.723091 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-e8defbc1-dae0-4346-acb3-595d9c6842c3 None None] MFA Rules not processed for user `cc99c93da5074cf5a48bc2aa9a92c5e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:59:44.724317 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-e8defbc1-dae0-4346-acb3-595d9c6842c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:59:44.731300 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-871d6128-8f2e-4a85-874d-a91776559be4 None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:44.733901 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:59:44.772533 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9175b76-b9ce-43d8-a435-9905aba0913b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:44.773221 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9175b76-b9ce-43d8-a435-9905aba0913b None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:44.773504 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9175b76-b9ce-43d8-a435-9905aba0913b None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:44.774236 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a9175b76-b9ce-43d8-a435-9905aba0913b None admin] PATH_INFO: `/v3/users` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:44.775381 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9175b76-b9ce-43d8-a435-9905aba0913b None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 21:59:44.775994 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9175b76-b9ce-43d8-a435-9905aba0913b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:44.777067 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a9175b76-b9ce-43d8-a435-9905aba0913b None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:44.796522 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 367/738] 10.222.0.52 () {56 vars in 1135 bytes} [Wed Jul 13 21:59:44 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 201 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 21:59:44.797360 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9175b76-b9ce-43d8-a435-9905aba0913b None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:44.801450 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 372/739] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jul 13 21:59:44 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 13 21:59:44.802403 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1bd4c5-60b6-4070-84f8-3242a657e994 None None] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:44.802720 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1bd4c5-60b6-4070-84f8-3242a657e994 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:44.802992 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc1bd4c5-60b6-4070-84f8-3242a657e994 None None] PATH_INFO: `/` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:44.808401 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 373/740] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jul 13 21:59:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 21:59:44.808401 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1bd4c5-60b6-4070-84f8-3242a657e994 None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:44.908669 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a9175b76-b9ce-43d8-a435-9905aba0913b None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:44.913777 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:59:44.922054 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc1bd4c5-60b6-4070-84f8-3242a657e994 None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:44.945673 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae0523bb-8b90-4e95-8d95-4b4641e90676 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:44.946396 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0523bb-8b90-4e95-8d95-4b4641e90676 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:44.946701 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0523bb-8b90-4e95-8d95-4b4641e90676 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:44.947070 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae0523bb-8b90-4e95-8d95-4b4641e90676 None admin] PATH_INFO: `/v3/users` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:44.947815 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0523bb-8b90-4e95-8d95-4b4641e90676 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 21:59:44.948431 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0523bb-8b90-4e95-8d95-4b4641e90676 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:44.949888 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae0523bb-8b90-4e95-8d95-4b4641e90676 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:44.962934 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e71a5cc-2e0e-44ae-b12d-ee7a4e7a7885 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:44.963617 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e71a5cc-2e0e-44ae-b12d-ee7a4e7a7885 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:44.963902 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e71a5cc-2e0e-44ae-b12d-ee7a4e7a7885 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:44.964180 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e71a5cc-2e0e-44ae-b12d-ee7a4e7a7885 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:44.971291 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 368/741] 10.222.0.52 () {56 vars in 1149 bytes} [Wed Jul 13 21:59:44 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 21:59:44.977067 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156c466d-8284-466f-ab09-db13b9c5b1fe None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:44.977346 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156c466d-8284-466f-ab09-db13b9c5b1fe None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:44.978002 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-156c466d-8284-466f-ab09-db13b9c5b1fe None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:44.995142 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-156c466d-8284-466f-ab09-db13b9c5b1fe None None] MFA Rules not processed for user `cc99c93da5074cf5a48bc2aa9a92c5e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:59:44.996333 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-156c466d-8284-466f-ab09-db13b9c5b1fe None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:59:45.012381 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:59:45.039297 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e71a5cc-2e0e-44ae-b12d-ee7a4e7a7885 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:45.040885 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e71a5cc-2e0e-44ae-b12d-ee7a4e7a7885 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:45.082599 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 369/742] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jul 13 21:59:44 2022] POST /identity/v3/auth/tokens => generated 903 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 363 bytes (1 switches on core 0) Jul 13 21:59:45.082599 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156c466d-8284-466f-ab09-db13b9c5b1fe None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:45.152416 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:59:45.164920 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 374/743] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Jul 13 21:59:44 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 13 21:59:45.164920 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e71a5cc-2e0e-44ae-b12d-ee7a4e7a7885 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:45.183306 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-156c466d-8284-466f-ab09-db13b9c5b1fe None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:45.223421 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9325f3e-0904-4a0a-bc0d-c7096d202996 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:45.224234 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9325f3e-0904-4a0a-bc0d-c7096d202996 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:45.224513 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9325f3e-0904-4a0a-bc0d-c7096d202996 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:45.224802 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9325f3e-0904-4a0a-bc0d-c7096d202996 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:45.235820 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9325f3e-0904-4a0a-bc0d-c7096d202996 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 21:59:45.243759 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application [None req-b9325f3e-0904-4a0a-bc0d-c7096d202996 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 13 21:59:45.243759 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 21:59:45.243759 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 21:59:45.243759 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 21:59:45.243759 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 21:59:45.243759 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 21:59:45.243759 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 21:59:45.243759 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 21:59:45.243759 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 21:59:45.243759 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 21:59:45.243759 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 21:59:45.243759 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 21:59:45.243759 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 21:59:45.243759 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 21:59:45.243759 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 21:59:45.243759 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 21:59:45.244715 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 21:59:45.244715 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 21:59:45.244715 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 21:59:45.244715 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 21:59:45.244715 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 21:59:45.244715 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 21:59:45.244715 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 21:59:45.244715 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 21:59:45.244715 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 21:59:45.244715 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 21:59:45.244715 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 21:59:45.244715 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application with Lock( Jul 13 21:59:45.244715 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 21:59:45.244715 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._enter() Jul 13 21:59:45.244715 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 21:59:45.244715 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 21:59:45.245579 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 21:59:45.245579 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.creator() Jul 13 21:59:45.245579 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 21:59:45.245579 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application created_value = creator( Jul 13 21:59:45.245579 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 21:59:45.245579 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 21:59:45.245579 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 21:59:45.245579 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 21:59:45.245579 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 21:59:45.245579 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 21:59:45.245579 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 13 21:59:45.245579 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Jul 13 21:59:45.245579 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 370/744] 10.222.0.52 () {56 vars in 1096 bytes} [Wed Jul 13 21:59:45 2022] GET /identity/v3/roles/admin => generated 83 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 21:59:45.246275 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9325f3e-0904-4a0a-bc0d-c7096d202996 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:45.262718 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:59:45.270261 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e71a5cc-2e0e-44ae-b12d-ee7a4e7a7885 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:45.307829 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9b0718-b2df-4ba1-874c-e2796f7556be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:45.308489 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9b0718-b2df-4ba1-874c-e2796f7556be None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:45.308775 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9b0718-b2df-4ba1-874c-e2796f7556be None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:45.309036 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9b0718-b2df-4ba1-874c-e2796f7556be None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:45.359781 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9325f3e-0904-4a0a-bc0d-c7096d202996 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:45.372171 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:59:45.380483 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9b0718-b2df-4ba1-874c-e2796f7556be None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:45.382029 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d9b0718-b2df-4ba1-874c-e2796f7556be None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:45.393796 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40855f9f-6d85-4fb0-be89-5f8c87c6591a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:45.394468 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40855f9f-6d85-4fb0-be89-5f8c87c6591a None admin] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:45.394767 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40855f9f-6d85-4fb0-be89-5f8c87c6591a None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:45.395040 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40855f9f-6d85-4fb0-be89-5f8c87c6591a None admin] PATH_INFO: `/v3/services` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:45.395771 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40855f9f-6d85-4fb0-be89-5f8c87c6591a None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:45.397125 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40855f9f-6d85-4fb0-be89-5f8c87c6591a None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:45.413544 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 371/745] 10.222.0.52 () {60 vars in 1141 bytes} [Wed Jul 13 21:59:45 2022] POST /identity/v3/services => generated 240 bytes in 168 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 21:59:45.414578 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40855f9f-6d85-4fb0-be89-5f8c87c6591a None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:45.498943 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 375/746] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Jul 13 21:59:45 2022] GET /identity/v3/auth/tokens => generated 903 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 358 bytes (1 switches on core 0) Jul 13 21:59:45.500191 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9b0718-b2df-4ba1-874c-e2796f7556be None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:45.526881 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:59:45.566886 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40855f9f-6d85-4fb0-be89-5f8c87c6591a None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:45.614474 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:59:45.615892 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c7fae2-c447-47e5-b6b5-4c3ad2f50e62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:45.616821 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c7fae2-c447-47e5-b6b5-4c3ad2f50e62 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:45.617153 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c7fae2-c447-47e5-b6b5-4c3ad2f50e62 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:45.617807 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c7fae2-c447-47e5-b6b5-4c3ad2f50e62 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:45.621571 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9b0718-b2df-4ba1-874c-e2796f7556be None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:45.631576 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c7fae2-c447-47e5-b6b5-4c3ad2f50e62 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:45.633326 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c7fae2-c447-47e5-b6b5-4c3ad2f50e62 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:45.653582 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application [None req-97c7fae2-c447-47e5-b6b5-4c3ad2f50e62 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 21:59:45.653582 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 21:59:45.653582 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 21:59:45.653582 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 21:59:45.653582 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 21:59:45.653582 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 21:59:45.653582 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 21:59:45.653582 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 21:59:45.653582 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 21:59:45.653582 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 21:59:45.653582 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 21:59:45.653582 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 21:59:45.653582 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 21:59:45.653582 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 21:59:45.653582 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 21:59:45.654649 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 21:59:45.654649 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 21:59:45.654649 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 21:59:45.654649 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 21:59:45.654649 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 21:59:45.654649 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 21:59:45.654649 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 21:59:45.654649 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 21:59:45.654649 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application with Lock( Jul 13 21:59:45.654649 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 21:59:45.654649 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._enter() Jul 13 21:59:45.654649 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 21:59:45.654649 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 21:59:45.654649 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 21:59:45.654649 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.creator() Jul 13 21:59:45.654649 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 21:59:45.654649 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application created_value = creator( Jul 13 21:59:45.655551 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 21:59:45.655551 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 21:59:45.655551 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 21:59:45.655551 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Jul 13 21:59:45.655551 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 372/747] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Jul 13 21:59:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 241 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 21:59:45.656709 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c7fae2-c447-47e5-b6b5-4c3ad2f50e62 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:45.670550 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3388e811-49fb-48db-a5b4-6a6138712243 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:45.671381 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3388e811-49fb-48db-a5b4-6a6138712243 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:45.671598 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3388e811-49fb-48db-a5b4-6a6138712243 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:45.671763 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3388e811-49fb-48db-a5b4-6a6138712243 None admin] PATH_INFO: `/v3/roles` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:45.672531 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3388e811-49fb-48db-a5b4-6a6138712243 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 21:59:45.672879 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3388e811-49fb-48db-a5b4-6a6138712243 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:45.674236 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3388e811-49fb-48db-a5b4-6a6138712243 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:45.683791 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 376/748] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Jul 13 21:59:45 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 184 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 21:59:45.684697 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3388e811-49fb-48db-a5b4-6a6138712243 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:45.764444 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c7fae2-c447-47e5-b6b5-4c3ad2f50e62 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:45.782550 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:59:45.793962 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3388e811-49fb-48db-a5b4-6a6138712243 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:45.801946 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f84264-c840-4bf7-b87b-61d088e8301e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:45.803144 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f84264-c840-4bf7-b87b-61d088e8301e None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:45.803625 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f84264-c840-4bf7-b87b-61d088e8301e None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:45.804094 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5f84264-c840-4bf7-b87b-61d088e8301e None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:45.816655 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f84264-c840-4bf7-b87b-61d088e8301e None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:45.818427 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5f84264-c840-4bf7-b87b-61d088e8301e None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:45.830012 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97632701-eff0-4f6d-8395-7e85b9a3646b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:45.830981 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97632701-eff0-4f6d-8395-7e85b9a3646b None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:45.831375 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97632701-eff0-4f6d-8395-7e85b9a3646b None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:45.831787 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97632701-eff0-4f6d-8395-7e85b9a3646b None admin] PATH_INFO: `/v3/domains` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:45.832623 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97632701-eff0-4f6d-8395-7e85b9a3646b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 21:59:45.833031 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97632701-eff0-4f6d-8395-7e85b9a3646b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:45.834546 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97632701-eff0-4f6d-8395-7e85b9a3646b None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:45.838825 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application [None req-e5f84264-c840-4bf7-b87b-61d088e8301e None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 21:59:45.838825 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 21:59:45.838825 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 21:59:45.838825 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 21:59:45.838825 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 21:59:45.838825 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 21:59:45.838825 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 21:59:45.838825 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 21:59:45.838825 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 21:59:45.838825 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 21:59:45.838825 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 21:59:45.838825 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 21:59:45.838825 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 21:59:45.838825 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 21:59:45.838825 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 21:59:45.839888 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 21:59:45.839888 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 21:59:45.839888 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 21:59:45.839888 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 21:59:45.839888 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 21:59:45.839888 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 21:59:45.839888 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 21:59:45.839888 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 21:59:45.839888 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application with Lock( Jul 13 21:59:45.839888 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 21:59:45.839888 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._enter() Jul 13 21:59:45.839888 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 21:59:45.839888 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 21:59:45.839888 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 21:59:45.839888 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.creator() Jul 13 21:59:45.839888 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 21:59:45.839888 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application created_value = creator( Jul 13 21:59:45.841013 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 21:59:45.841013 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 21:59:45.841013 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 21:59:45.841013 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Jul 13 21:59:45.843393 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 373/749] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Jul 13 21:59:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 188 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 21:59:45.844434 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f84264-c840-4bf7-b87b-61d088e8301e None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:45.850683 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 377/750] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Jul 13 21:59:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 21:59:45.851697 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97632701-eff0-4f6d-8395-7e85b9a3646b None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:45.876886 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:59:45.965487 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5f84264-c840-4bf7-b87b-61d088e8301e None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:45.969637 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97632701-eff0-4f6d-8395-7e85b9a3646b None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:46.001132 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea54d58-797a-43a5-9aee-84403db57be5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:46.001949 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea54d58-797a-43a5-9aee-84403db57be5 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:46.002366 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea54d58-797a-43a5-9aee-84403db57be5 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:46.002793 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ea54d58-797a-43a5-9aee-84403db57be5 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:46.009467 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e233b90-e49d-4532-8d95-ea2cf6dbe307 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:46.010411 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e233b90-e49d-4532-8d95-ea2cf6dbe307 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:46.010969 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e233b90-e49d-4532-8d95-ea2cf6dbe307 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:46.011439 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e233b90-e49d-4532-8d95-ea2cf6dbe307 None admin] PATH_INFO: `/v3/domains` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:46.012774 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e233b90-e49d-4532-8d95-ea2cf6dbe307 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 21:59:46.013285 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e233b90-e49d-4532-8d95-ea2cf6dbe307 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:46.014987 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e233b90-e49d-4532-8d95-ea2cf6dbe307 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:46.019181 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea54d58-797a-43a5-9aee-84403db57be5 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:46.020633 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ea54d58-797a-43a5-9aee-84403db57be5 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:46.027604 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 378/751] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Jul 13 21:59:45 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 21:59:46.028811 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e233b90-e49d-4532-8d95-ea2cf6dbe307 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:46.047144 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application [None req-5ea54d58-797a-43a5-9aee-84403db57be5 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 21:59:46.047144 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 21:59:46.047144 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 21:59:46.047144 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 21:59:46.047144 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 21:59:46.047144 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 21:59:46.047144 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 21:59:46.047144 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 21:59:46.047144 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 21:59:46.047144 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 21:59:46.047144 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 21:59:46.047144 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 21:59:46.047144 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 21:59:46.047144 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 21:59:46.047144 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 21:59:46.048091 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 21:59:46.048091 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 21:59:46.048091 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 21:59:46.048091 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 21:59:46.048091 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 21:59:46.048091 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 21:59:46.048091 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 21:59:46.048091 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 21:59:46.048091 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application with Lock( Jul 13 21:59:46.048091 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 21:59:46.048091 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._enter() Jul 13 21:59:46.048091 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 21:59:46.048091 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 21:59:46.048091 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 21:59:46.048091 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.creator() Jul 13 21:59:46.048091 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 21:59:46.048091 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application created_value = creator( Jul 13 21:59:46.048983 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 21:59:46.048983 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 21:59:46.048983 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 21:59:46.048983 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Jul 13 21:59:46.050640 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 374/752] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Jul 13 21:59:45 2022] GET /identity/v3/domains/Default => generated 87 bytes in 207 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 21:59:46.051650 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea54d58-797a-43a5-9aee-84403db57be5 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:46.081594 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:59:46.130175 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:59:46.153604 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e233b90-e49d-4532-8d95-ea2cf6dbe307 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:46.161070 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ea54d58-797a-43a5-9aee-84403db57be5 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:46.192212 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d3616d-21a6-4bb5-8642-1f841ba0702b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:46.193097 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d3616d-21a6-4bb5-8642-1f841ba0702b None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:46.193382 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d3616d-21a6-4bb5-8642-1f841ba0702b None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:46.193654 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16d3616d-21a6-4bb5-8642-1f841ba0702b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:46.197152 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9e93f9-40aa-4be2-8f6e-0eadce6e472b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:46.198703 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9e93f9-40aa-4be2-8f6e-0eadce6e472b None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:46.199524 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9e93f9-40aa-4be2-8f6e-0eadce6e472b None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:46.199993 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df9e93f9-40aa-4be2-8f6e-0eadce6e472b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:46.201283 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d3616d-21a6-4bb5-8642-1f841ba0702b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:46.202894 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16d3616d-21a6-4bb5-8642-1f841ba0702b None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:46.208132 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df9e93f9-40aa-4be2-8f6e-0eadce6e472b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:46.209797 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df9e93f9-40aa-4be2-8f6e-0eadce6e472b None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:46.220105 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application [None req-16d3616d-21a6-4bb5-8642-1f841ba0702b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 13 21:59:46.220105 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 21:59:46.220105 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 21:59:46.220105 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 21:59:46.220105 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 21:59:46.220105 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 21:59:46.220105 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 21:59:46.220105 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 21:59:46.220105 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 21:59:46.220105 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 21:59:46.220105 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 21:59:46.220105 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 21:59:46.220105 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 13 21:59:46.220105 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 13 21:59:46.220105 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 13 21:59:46.221053 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 13 21:59:46.221053 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 21:59:46.221053 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 21:59:46.221053 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 21:59:46.221053 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 21:59:46.221053 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 21:59:46.221053 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 21:59:46.221053 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 21:59:46.221053 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application with Lock( Jul 13 21:59:46.221053 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 21:59:46.221053 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._enter() Jul 13 21:59:46.221053 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 21:59:46.221053 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 21:59:46.221053 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 21:59:46.221053 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.creator() Jul 13 21:59:46.221053 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 21:59:46.221053 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application created_value = creator( Jul 13 21:59:46.221949 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 13 21:59:46.221949 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 13 21:59:46.221949 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 13 21:59:46.221949 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 13 21:59:46.221949 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 13 21:59:46.221949 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 13 21:59:46.221949 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 13 21:59:46.221949 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Jul 13 21:59:46.226296 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 379/753] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Jul 13 21:59:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 21:59:46.227146 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d3616d-21a6-4bb5-8642-1f841ba0702b None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:46.227687 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application [None req-df9e93f9-40aa-4be2-8f6e-0eadce6e472b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 13 21:59:46.227687 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 21:59:46.227687 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 21:59:46.227687 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 21:59:46.227687 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 21:59:46.227687 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 21:59:46.227687 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 21:59:46.227687 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 21:59:46.227687 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 21:59:46.227687 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 21:59:46.227687 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 21:59:46.227687 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 21:59:46.227687 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 13 21:59:46.227687 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 13 21:59:46.227687 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 13 21:59:46.228703 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 13 21:59:46.228703 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 21:59:46.228703 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 21:59:46.228703 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 21:59:46.228703 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 21:59:46.228703 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 21:59:46.228703 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 21:59:46.228703 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 21:59:46.228703 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application with Lock( Jul 13 21:59:46.228703 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 21:59:46.228703 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._enter() Jul 13 21:59:46.228703 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 21:59:46.228703 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 21:59:46.228703 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 21:59:46.228703 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.creator() Jul 13 21:59:46.228703 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 21:59:46.228703 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application created_value = creator( Jul 13 21:59:46.229585 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 13 21:59:46.229585 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 13 21:59:46.229585 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 13 21:59:46.229585 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 13 21:59:46.229585 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 13 21:59:46.229585 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 13 21:59:46.229585 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 13 21:59:46.229585 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Jul 13 21:59:46.231996 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 375/754] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Jul 13 21:59:46 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 21:59:46.233039 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9e93f9-40aa-4be2-8f6e-0eadce6e472b None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:46.323468 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:59:46.343072 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df9e93f9-40aa-4be2-8f6e-0eadce6e472b None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:46.343567 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16d3616d-21a6-4bb5-8642-1f841ba0702b None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:46.379304 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc9176b-e3fe-415c-ae98-0854fc90d17c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:46.380372 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc9176b-e3fe-415c-ae98-0854fc90d17c None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:46.380824 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc9176b-e3fe-415c-ae98-0854fc90d17c None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:46.381261 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5821c2e6-5b51-4053-bbd4-8b6a5219771f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:46.381492 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dc9176b-e3fe-415c-ae98-0854fc90d17c None admin] PATH_INFO: `/v3/projects` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:46.382050 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5821c2e6-5b51-4053-bbd4-8b6a5219771f None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:46.382410 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5821c2e6-5b51-4053-bbd4-8b6a5219771f None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:46.382785 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5821c2e6-5b51-4053-bbd4-8b6a5219771f None admin] PATH_INFO: `/v3/domains` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:46.383062 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc9176b-e3fe-415c-ae98-0854fc90d17c None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 21:59:46.383538 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc9176b-e3fe-415c-ae98-0854fc90d17c None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:46.383665 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5821c2e6-5b51-4053-bbd4-8b6a5219771f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 21:59:46.384190 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5821c2e6-5b51-4053-bbd4-8b6a5219771f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:46.385478 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dc9176b-e3fe-415c-ae98-0854fc90d17c None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:46.385828 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5821c2e6-5b51-4053-bbd4-8b6a5219771f None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:46.392923 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:59:46.400502 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 376/755] 10.222.0.52 () {56 vars in 1148 bytes} [Wed Jul 13 21:59:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 21:59:46.401574 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc9176b-e3fe-415c-ae98-0854fc90d17c None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:46.402640 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 380/756] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Jul 13 21:59:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 21:59:46.409097 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5821c2e6-5b51-4053-bbd4-8b6a5219771f None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:46.506376 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dc9176b-e3fe-415c-ae98-0854fc90d17c None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:46.517307 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5821c2e6-5b51-4053-bbd4-8b6a5219771f None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:46.541242 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e4096b-515f-4601-9130-cbade6b40658 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:46.541919 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e4096b-515f-4601-9130-cbade6b40658 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:46.542201 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e4096b-515f-4601-9130-cbade6b40658 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:46.542479 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60e4096b-515f-4601-9130-cbade6b40658 None admin] PATH_INFO: `/v3/projects` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:46.543316 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e4096b-515f-4601-9130-cbade6b40658 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 21:59:46.543672 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e4096b-515f-4601-9130-cbade6b40658 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:46.545120 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60e4096b-515f-4601-9130-cbade6b40658 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:46.557544 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4c5291-2cdd-4964-b09d-a3b86490c6e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:46.562545 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 377/757] 10.222.0.52 () {56 vars in 1148 bytes} [Wed Jul 13 21:59:46 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 21:59:46.562545 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e4096b-515f-4601-9130-cbade6b40658 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:46.563396 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4c5291-2cdd-4964-b09d-a3b86490c6e9 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:46.563396 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4c5291-2cdd-4964-b09d-a3b86490c6e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:46.563657 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f4c5291-2cdd-4964-b09d-a3b86490c6e9 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:46.567845 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:59:46.574431 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4c5291-2cdd-4964-b09d-a3b86490c6e9 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:46.575941 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f4c5291-2cdd-4964-b09d-a3b86490c6e9 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:46.594197 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application [None req-8f4c5291-2cdd-4964-b09d-a3b86490c6e9 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 13 21:59:46.594197 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 21:59:46.594197 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 21:59:46.594197 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 21:59:46.594197 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 21:59:46.594197 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 21:59:46.594197 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 21:59:46.594197 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 21:59:46.594197 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 21:59:46.594197 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 21:59:46.594197 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 21:59:46.594197 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 21:59:46.594197 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 13 21:59:46.594197 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 13 21:59:46.594197 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 13 21:59:46.594197 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 13 21:59:46.595256 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 21:59:46.595256 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 21:59:46.595256 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 13 21:59:46.595256 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 21:59:46.595256 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 13 21:59:46.595256 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 21:59:46.595256 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 21:59:46.595256 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 21:59:46.595256 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 21:59:46.595256 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 21:59:46.595256 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 21:59:46.595256 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application with Lock( Jul 13 21:59:46.595256 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 21:59:46.595256 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._enter() Jul 13 21:59:46.595256 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 21:59:46.595256 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 21:59:46.595256 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 21:59:46.595256 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.creator() Jul 13 21:59:46.596194 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 21:59:46.596194 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application created_value = creator( Jul 13 21:59:46.596194 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 13 21:59:46.596194 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 13 21:59:46.596194 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 13 21:59:46.596194 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 13 21:59:46.596194 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 13 21:59:46.596194 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 13 21:59:46.596194 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 13 21:59:46.596194 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Jul 13 21:59:46.596194 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 381/758] 10.222.0.52 () {56 vars in 1093 bytes} [Wed Jul 13 21:59:46 2022] GET /identity/v3/users/nova => generated 82 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 13 21:59:46.596790 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4c5291-2cdd-4964-b09d-a3b86490c6e9 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:46.671143 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:59:46.707890 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60e4096b-515f-4601-9130-cbade6b40658 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:46.721481 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f4c5291-2cdd-4964-b09d-a3b86490c6e9 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:46.742950 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7632f62a-5ea3-462e-b39b-252284464558 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:46.743680 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7632f62a-5ea3-462e-b39b-252284464558 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:46.743933 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7632f62a-5ea3-462e-b39b-252284464558 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:46.744202 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7632f62a-5ea3-462e-b39b-252284464558 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:46.745033 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7632f62a-5ea3-462e-b39b-252284464558 None admin] RBAC: Adding query filter params (role.id=44efba2579f342df8a0789c59d163ec3, scope.project.id=417e3e2f1333481795a089bef2e24bea, user.id=1eb1525233ba48c99b15d4cb75db4822) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 21:59:46.745338 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7632f62a-5ea3-462e-b39b-252284464558 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:46.746781 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7632f62a-5ea3-462e-b39b-252284464558 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:46.752322 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 378/759] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Jul 13 21:59:46 2022] GET /identity/v3/role_assignments?user.id=1eb1525233ba48c99b15d4cb75db4822&scope.project.id=417e3e2f1333481795a089bef2e24bea&role.id=44efba2579f342df8a0789c59d163ec3 => generated 259 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 21:59:46.753076 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7632f62a-5ea3-462e-b39b-252284464558 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:46.781841 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142a49f1-90cd-43b9-9d61-0fca2dfb545f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:46.783168 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142a49f1-90cd-43b9-9d61-0fca2dfb545f None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:46.783621 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142a49f1-90cd-43b9-9d61-0fca2dfb545f None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:46.784333 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142a49f1-90cd-43b9-9d61-0fca2dfb545f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:46.785897 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142a49f1-90cd-43b9-9d61-0fca2dfb545f None admin] RBAC: Adding query filter params (role.id=70c34ea493484322911dab62b4a56e75, scope.project.id=417e3e2f1333481795a089bef2e24bea, user.id=b866bacdafae4db6a32714c07c56515f) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 21:59:46.786672 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142a49f1-90cd-43b9-9d61-0fca2dfb545f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:46.789121 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142a49f1-90cd-43b9-9d61-0fca2dfb545f None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:46.798698 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 382/760] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Jul 13 21:59:46 2022] GET /identity/v3/role_assignments?user.id=b866bacdafae4db6a32714c07c56515f&scope.project.id=417e3e2f1333481795a089bef2e24bea&role.id=70c34ea493484322911dab62b4a56e75 => generated 259 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 21:59:46.800454 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacdcce4-9cab-45b7-b2c1-0598f68ef90d None None] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:46.800890 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacdcce4-9cab-45b7-b2c1-0598f68ef90d None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:46.801328 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bacdcce4-9cab-45b7-b2c1-0598f68ef90d None None] PATH_INFO: `/` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:46.803389 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 383/761] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jul 13 21:59:46 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 21:59:46.810833 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eda4080-6781-4ad1-9d86-ae07acef7a2f None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:46.811363 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eda4080-6781-4ad1-9d86-ae07acef7a2f None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:46.811811 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9eda4080-6781-4ad1-9d86-ae07acef7a2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:46.853926 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-9eda4080-6781-4ad1-9d86-ae07acef7a2f None None] MFA Rules not processed for user `cc99c93da5074cf5a48bc2aa9a92c5e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:59:46.857476 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-9eda4080-6781-4ad1-9d86-ae07acef7a2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:59:46.881992 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:59:46.929526 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7632f62a-5ea3-462e-b39b-252284464558 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:46.999080 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40fae952-1afb-41a5-86f3-c7b083700eeb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:46.999867 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fae952-1afb-41a5-86f3-c7b083700eeb None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:47.000243 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fae952-1afb-41a5-86f3-c7b083700eeb None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:47.000430 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 384/762] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jul 13 21:59:46 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 13 21:59:47.000544 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40fae952-1afb-41a5-86f3-c7b083700eeb None admin] PATH_INFO: `/v3/users` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:47.002247 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40fae952-1afb-41a5-86f3-c7b083700eeb None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 21:59:47.002247 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40fae952-1afb-41a5-86f3-c7b083700eeb None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:47.003675 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40fae952-1afb-41a5-86f3-c7b083700eeb None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:47.031279 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eda4080-6781-4ad1-9d86-ae07acef7a2f None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:47.050113 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 379/763] 10.222.0.52 () {56 vars in 1133 bytes} [Wed Jul 13 21:59:46 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 21:59:47.053612 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40fae952-1afb-41a5-86f3-c7b083700eeb None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:47.130891 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:59:47.202235 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9eda4080-6781-4ad1-9d86-ae07acef7a2f None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:47.209268 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40fae952-1afb-41a5-86f3-c7b083700eeb None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:47.234173 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:59:47.246525 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69cc954c-5a4b-4553-8e96-fb5223400860 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:47.246525 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cc954c-5a4b-4553-8e96-fb5223400860 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:47.246525 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cc954c-5a4b-4553-8e96-fb5223400860 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:47.246525 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69cc954c-5a4b-4553-8e96-fb5223400860 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:47.247000 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69cc954c-5a4b-4553-8e96-fb5223400860 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:47.247084 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69cc954c-5a4b-4553-8e96-fb5223400860 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:47.262100 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application [None req-69cc954c-5a4b-4553-8e96-fb5223400860 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 13 21:59:47.262100 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 21:59:47.262100 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 13 21:59:47.262100 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 13 21:59:47.262100 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 13 21:59:47.262100 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 13 21:59:47.262100 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 13 21:59:47.262100 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 13 21:59:47.262100 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 13 21:59:47.262100 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Jul 13 21:59:47.262100 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 13 21:59:47.262100 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Jul 13 21:59:47.262100 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 21:59:47.262100 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 21:59:47.262100 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 21:59:47.262100 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 21:59:47.263296 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 21:59:47.263296 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 21:59:47.263296 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 21:59:47.263296 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 21:59:47.263296 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 21:59:47.263296 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 21:59:47.263296 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 21:59:47.263296 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 13 21:59:47.263296 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 13 21:59:47.263296 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 13 21:59:47.263296 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 13 21:59:47.263296 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 21:59:47.263296 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 21:59:47.263296 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 21:59:47.263296 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 21:59:47.263296 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 21:59:47.264350 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 21:59:47.264350 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 21:59:47.264350 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application with Lock( Jul 13 21:59:47.264350 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 21:59:47.264350 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._enter() Jul 13 21:59:47.264350 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 21:59:47.264350 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 21:59:47.264350 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 21:59:47.264350 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.creator() Jul 13 21:59:47.264350 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 21:59:47.264350 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application created_value = creator( Jul 13 21:59:47.264350 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 13 21:59:47.264350 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 13 21:59:47.264350 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 13 21:59:47.264350 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Jul 13 21:59:47.266130 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d91b587-9779-4b47-b9c7-71f6f8c10802 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:47.266829 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 385/764] 10.222.0.52 () {56 vars in 1129 bytes} [Wed Jul 13 21:59:47 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 236 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 13 21:59:47.267294 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d91b587-9779-4b47-b9c7-71f6f8c10802 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:47.267705 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d91b587-9779-4b47-b9c7-71f6f8c10802 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:47.268113 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d91b587-9779-4b47-b9c7-71f6f8c10802 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:47.269721 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69cc954c-5a4b-4553-8e96-fb5223400860 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:47.280260 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d91b587-9779-4b47-b9c7-71f6f8c10802 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:47.281975 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d91b587-9779-4b47-b9c7-71f6f8c10802 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:47.301616 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application [None req-5d91b587-9779-4b47-b9c7-71f6f8c10802 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 21:59:47.301616 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 21:59:47.301616 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 21:59:47.301616 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 21:59:47.301616 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 21:59:47.301616 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 21:59:47.301616 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 21:59:47.301616 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 21:59:47.301616 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 21:59:47.301616 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 21:59:47.301616 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 21:59:47.301616 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 21:59:47.301616 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 21:59:47.301616 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 21:59:47.301616 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 21:59:47.302856 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 21:59:47.302856 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 21:59:47.302856 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 21:59:47.302856 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 21:59:47.302856 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 21:59:47.302856 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 21:59:47.302856 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 21:59:47.302856 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 21:59:47.302856 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application with Lock( Jul 13 21:59:47.302856 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 21:59:47.302856 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._enter() Jul 13 21:59:47.302856 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 21:59:47.302856 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 21:59:47.302856 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 21:59:47.302856 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.creator() Jul 13 21:59:47.302856 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 21:59:47.302856 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application created_value = creator( Jul 13 21:59:47.303775 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 21:59:47.303775 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 21:59:47.303775 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 21:59:47.303775 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Jul 13 21:59:47.309023 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 380/765] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Jul 13 21:59:47 2022] GET /identity/v3/domains/Default => generated 87 bytes in 253 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 21:59:47.310106 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d91b587-9779-4b47-b9c7-71f6f8c10802 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:47.418671 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d91b587-9779-4b47-b9c7-71f6f8c10802 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:47.455834 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366c8e8b-0b12-43d4-948e-13dec2d8a241 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:47.456857 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366c8e8b-0b12-43d4-948e-13dec2d8a241 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:47.457346 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366c8e8b-0b12-43d4-948e-13dec2d8a241 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:47.457808 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-366c8e8b-0b12-43d4-948e-13dec2d8a241 None admin] PATH_INFO: `/v3/domains` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:47.458764 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366c8e8b-0b12-43d4-948e-13dec2d8a241 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 21:59:47.459280 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366c8e8b-0b12-43d4-948e-13dec2d8a241 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:47.460868 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-366c8e8b-0b12-43d4-948e-13dec2d8a241 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:47.475079 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 381/766] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Jul 13 21:59:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 21:59:47.481082 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366c8e8b-0b12-43d4-948e-13dec2d8a241 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:47.520253 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:59:47.602928 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-366c8e8b-0b12-43d4-948e-13dec2d8a241 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:47.639207 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69cc954c-5a4b-4553-8e96-fb5223400860 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:47.642905 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9de4deb1-9abf-4a00-8991-039da9833aef None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:47.643956 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de4deb1-9abf-4a00-8991-039da9833aef None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:47.644428 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de4deb1-9abf-4a00-8991-039da9833aef None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:47.644889 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9de4deb1-9abf-4a00-8991-039da9833aef None admin] PATH_INFO: `/v3/projects/service` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:47.652849 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de4deb1-9abf-4a00-8991-039da9833aef None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:47.654732 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9de4deb1-9abf-4a00-8991-039da9833aef None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:47.660022 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:59:47.671745 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application [None req-9de4deb1-9abf-4a00-8991-039da9833aef None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 13 21:59:47.671745 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 21:59:47.671745 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 21:59:47.671745 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 21:59:47.671745 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 21:59:47.671745 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 21:59:47.671745 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 21:59:47.671745 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 21:59:47.671745 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 21:59:47.671745 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 21:59:47.671745 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 21:59:47.671745 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 21:59:47.671745 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 13 21:59:47.671745 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 13 21:59:47.671745 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 13 21:59:47.672657 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 13 21:59:47.672657 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 21:59:47.672657 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 21:59:47.672657 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 21:59:47.672657 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 21:59:47.672657 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 21:59:47.672657 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 21:59:47.672657 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 21:59:47.672657 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application with Lock( Jul 13 21:59:47.672657 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 21:59:47.672657 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._enter() Jul 13 21:59:47.672657 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 21:59:47.672657 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 21:59:47.672657 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 21:59:47.672657 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.creator() Jul 13 21:59:47.672657 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 21:59:47.672657 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application created_value = creator( Jul 13 21:59:47.673552 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 13 21:59:47.673552 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 13 21:59:47.673552 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 13 21:59:47.673552 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 13 21:59:47.673552 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 13 21:59:47.673552 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 13 21:59:47.673552 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 13 21:59:47.673552 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Jul 13 21:59:47.677828 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 382/767] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Jul 13 21:59:47 2022] GET /identity/v3/projects/service => generated 88 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 13 21:59:47.679479 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5124c1fc-9e9a-4993-ac15-13a0228efa12 None None] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:47.679768 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5124c1fc-9e9a-4993-ac15-13a0228efa12 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:47.680037 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5124c1fc-9e9a-4993-ac15-13a0228efa12 None None] PATH_INFO: `/` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:47.681739 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 383/768] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jul 13 21:59:47 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 21:59:47.682803 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5124c1fc-9e9a-4993-ac15-13a0228efa12 None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:47.683114 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d73d3ab-f959-44dd-97ea-70854f699183 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:47.684021 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d73d3ab-f959-44dd-97ea-70854f699183 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:47.684562 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d73d3ab-f959-44dd-97ea-70854f699183 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:47.685027 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d73d3ab-f959-44dd-97ea-70854f699183 None admin] PATH_INFO: `/v3/services` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:47.686128 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d73d3ab-f959-44dd-97ea-70854f699183 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 21:59:47.686723 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d73d3ab-f959-44dd-97ea-70854f699183 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:47.688367 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d73d3ab-f959-44dd-97ea-70854f699183 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:47.693892 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 386/769] 10.222.0.52 () {56 vars in 1124 bytes} [Wed Jul 13 21:59:47 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 425 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 21:59:47.700564 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4bfeaa-e68e-4cdb-a3c8-b901d301ef4c None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:47.701127 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4bfeaa-e68e-4cdb-a3c8-b901d301ef4c None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:47.701622 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e4bfeaa-e68e-4cdb-a3c8-b901d301ef4c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:47.720002 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-0e4bfeaa-e68e-4cdb-a3c8-b901d301ef4c None None] MFA Rules not processed for user `cc99c93da5074cf5a48bc2aa9a92c5e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:59:47.721617 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-0e4bfeaa-e68e-4cdb-a3c8-b901d301ef4c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:59:47.785583 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:59:47.793355 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 387/770] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jul 13 21:59:47 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 13 21:59:47.794570 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5124c1fc-9e9a-4993-ac15-13a0228efa12 None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:47.795004 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4bfeaa-e68e-4cdb-a3c8-b901d301ef4c None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:47.833782 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd10ef5-a836-43d3-a790-0a7880427c82 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:47.834563 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd10ef5-a836-43d3-a790-0a7880427c82 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:47.834894 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd10ef5-a836-43d3-a790-0a7880427c82 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:47.835166 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fd10ef5-a836-43d3-a790-0a7880427c82 None admin] PATH_INFO: `/v3/projects` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:47.836013 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd10ef5-a836-43d3-a790-0a7880427c82 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 21:59:47.836553 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd10ef5-a836-43d3-a790-0a7880427c82 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:47.839501 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fd10ef5-a836-43d3-a790-0a7880427c82 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:47.853712 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 384/771] 10.222.0.52 () {56 vars in 1148 bytes} [Wed Jul 13 21:59:47 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 21:59:47.859509 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd10ef5-a836-43d3-a790-0a7880427c82 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:47.901618 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e4bfeaa-e68e-4cdb-a3c8-b901d301ef4c None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:47.913123 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:59:47.938287 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9695f003-d9de-4a01-8cff-2260a3da125c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:47.939327 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9695f003-d9de-4a01-8cff-2260a3da125c None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:47.939802 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9695f003-d9de-4a01-8cff-2260a3da125c None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:47.940362 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9695f003-d9de-4a01-8cff-2260a3da125c None admin] PATH_INFO: `/v3/services` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:47.941364 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9695f003-d9de-4a01-8cff-2260a3da125c None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 21:59:47.941880 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9695f003-d9de-4a01-8cff-2260a3da125c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:47.943539 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9695f003-d9de-4a01-8cff-2260a3da125c None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:47.954176 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 388/772] 10.222.0.52 () {56 vars in 1124 bytes} [Wed Jul 13 21:59:47 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 21:59:47.955397 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9695f003-d9de-4a01-8cff-2260a3da125c None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:47.974155 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fd10ef5-a836-43d3-a790-0a7880427c82 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:48.014054 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94daafc-10fd-4b18-b57f-faf48b776986 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:48.014912 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94daafc-10fd-4b18-b57f-faf48b776986 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:48.015189 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94daafc-10fd-4b18-b57f-faf48b776986 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:48.015633 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b94daafc-10fd-4b18-b57f-faf48b776986 None admin] PATH_INFO: `/v3/services/network` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:48.016293 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94daafc-10fd-4b18-b57f-faf48b776986 None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:48.017818 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b94daafc-10fd-4b18-b57f-faf48b776986 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:48.035872 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application [None req-b94daafc-10fd-4b18-b57f-faf48b776986 None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 13 21:59:48.035872 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 21:59:48.035872 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 13 21:59:48.035872 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 13 21:59:48.035872 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 13 21:59:48.035872 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 13 21:59:48.035872 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 13 21:59:48.035872 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 13 21:59:48.035872 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 13 21:59:48.035872 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Jul 13 21:59:48.035872 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 13 21:59:48.035872 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Jul 13 21:59:48.035872 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 21:59:48.035872 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 21:59:48.035872 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 21:59:48.035872 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 21:59:48.035872 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 21:59:48.036898 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 21:59:48.036898 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 21:59:48.036898 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 21:59:48.036898 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 21:59:48.036898 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 21:59:48.036898 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 21:59:48.036898 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 13 21:59:48.036898 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 13 21:59:48.036898 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 13 21:59:48.036898 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 13 21:59:48.036898 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 21:59:48.036898 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 21:59:48.036898 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 21:59:48.036898 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 21:59:48.036898 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 21:59:48.036898 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 21:59:48.037750 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 21:59:48.037750 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application with Lock( Jul 13 21:59:48.037750 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 21:59:48.037750 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._enter() Jul 13 21:59:48.037750 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 21:59:48.037750 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 21:59:48.037750 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 21:59:48.037750 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.creator() Jul 13 21:59:48.037750 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 21:59:48.037750 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application created_value = creator( Jul 13 21:59:48.037750 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 13 21:59:48.037750 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 13 21:59:48.037750 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 13 21:59:48.037750 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Jul 13 21:59:48.037750 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 385/773] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Jul 13 21:59:47 2022] GET /identity/v3/services/network => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 21:59:48.038579 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94daafc-10fd-4b18-b57f-faf48b776986 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:48.065296 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9695f003-d9de-4a01-8cff-2260a3da125c None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:48.066901 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:59:48.101309 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1a198b-65cf-4e56-b394-60fc7ee42f9f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:48.102653 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1a198b-65cf-4e56-b394-60fc7ee42f9f None admin] REQUEST_METHOD: `PUT` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:48.103350 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1a198b-65cf-4e56-b394-60fc7ee42f9f None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:48.103822 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa1a198b-65cf-4e56-b394-60fc7ee42f9f None admin] PATH_INFO: `/v3/projects/417e3e2f1333481795a089bef2e24bea/users/f84281091687464990ffbca590b9343b/roles/44efba2579f342df8a0789c59d163ec3` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:48.106091 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa1a198b-65cf-4e56-b394-60fc7ee42f9f None admin] RBAC: Authorizing `identity:create_grant(project_id=417e3e2f1333481795a089bef2e24bea, user_id=f84281091687464990ffbca590b9343b, role_id=44efba2579f342df8a0789c59d163ec3)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:48.107802 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa1a198b-65cf-4e56-b394-60fc7ee42f9f None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:48.125985 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 389/774] 10.222.0.52 () {58 vars in 1439 bytes} [Wed Jul 13 21:59:47 2022] PUT /identity/v3/projects/417e3e2f1333481795a089bef2e24bea/users/f84281091687464990ffbca590b9343b/roles/44efba2579f342df8a0789c59d163ec3 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 21:59:48.127374 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1a198b-65cf-4e56-b394-60fc7ee42f9f None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:48.178521 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:59:48.193227 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b94daafc-10fd-4b18-b57f-faf48b776986 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:48.288397 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa1a198b-65cf-4e56-b394-60fc7ee42f9f None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:48.295248 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b4af23-b030-4876-9899-45b27bf97d15 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:48.295949 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b4af23-b030-4876-9899-45b27bf97d15 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:48.296236 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b4af23-b030-4876-9899-45b27bf97d15 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:48.296510 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1b4af23-b030-4876-9899-45b27bf97d15 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:48.297334 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b4af23-b030-4876-9899-45b27bf97d15 None admin] RBAC: Adding query filter params (interface=public, service_id=a19e7ee4fd914b24ae7086657e353d9c, region_id=RegionOne) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 21:59:48.297645 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b4af23-b030-4876-9899-45b27bf97d15 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:48.299097 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1b4af23-b030-4876-9899-45b27bf97d15 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:48.319321 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 386/775] 10.222.0.52 () {56 vars in 1251 bytes} [Wed Jul 13 21:59:48 2022] GET /identity/v3/endpoints?service_id=a19e7ee4fd914b24ae7086657e353d9c&interface=public®ion_id=RegionOne => generated 194 bytes in 281 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 21:59:48.327226 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b4af23-b030-4876-9899-45b27bf97d15 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:48.346295 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfcd8fd7-9e55-4b6c-91af-3e33983f475b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:48.348338 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcd8fd7-9e55-4b6c-91af-3e33983f475b None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:48.348338 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcd8fd7-9e55-4b6c-91af-3e33983f475b None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:48.348338 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dfcd8fd7-9e55-4b6c-91af-3e33983f475b None admin] PATH_INFO: `/v3/services` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:48.348338 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfcd8fd7-9e55-4b6c-91af-3e33983f475b None admin] RBAC: Adding query filter params (name=network) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 21:59:48.348658 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfcd8fd7-9e55-4b6c-91af-3e33983f475b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:48.350020 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dfcd8fd7-9e55-4b6c-91af-3e33983f475b None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:48.358387 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 390/776] 10.222.0.52 () {56 vars in 1112 bytes} [Wed Jul 13 21:59:48 2022] GET /identity/v3/services?name=network => generated 124 bytes in 231 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 21:59:48.367502 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfcd8fd7-9e55-4b6c-91af-3e33983f475b None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:48.422620 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:59:48.454956 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1b4af23-b030-4876-9899-45b27bf97d15 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:48.474854 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:59:48.479403 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dfcd8fd7-9e55-4b6c-91af-3e33983f475b None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:48.508378 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550256e4-07d1-4e10-b6ff-94009516d78c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:48.509381 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550256e4-07d1-4e10-b6ff-94009516d78c None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:48.509797 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550256e4-07d1-4e10-b6ff-94009516d78c None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:48.512565 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-550256e4-07d1-4e10-b6ff-94009516d78c None admin] PATH_INFO: `/v3/services` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:48.513412 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550256e4-07d1-4e10-b6ff-94009516d78c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:48.515007 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-550256e4-07d1-4e10-b6ff-94009516d78c None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:48.525096 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 387/777] 10.222.0.52 () {56 vars in 1087 bytes} [Wed Jul 13 21:59:48 2022] GET /identity/v3/services => generated 772 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 21:59:48.533972 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95894861-b9f8-46fd-ad19-07a5862d3b6d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:48.535017 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95894861-b9f8-46fd-ad19-07a5862d3b6d None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:48.535463 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95894861-b9f8-46fd-ad19-07a5862d3b6d None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:48.535885 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95894861-b9f8-46fd-ad19-07a5862d3b6d None admin] PATH_INFO: `/v3/services` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:48.536763 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95894861-b9f8-46fd-ad19-07a5862d3b6d None admin] RBAC: Adding query filter params (type=network) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 21:59:48.537210 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95894861-b9f8-46fd-ad19-07a5862d3b6d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:48.538965 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95894861-b9f8-46fd-ad19-07a5862d3b6d None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:48.552100 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 391/778] 10.222.0.52 () {56 vars in 1112 bytes} [Wed Jul 13 21:59:48 2022] GET /identity/v3/services?type=network => generated 350 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 21:59:48.553940 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550256e4-07d1-4e10-b6ff-94009516d78c None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:48.664436 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-550256e4-07d1-4e10-b6ff-94009516d78c None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:48.702160 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a08af7bb-e589-4d4e-9155-bac5d8f36a30 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:48.703277 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08af7bb-e589-4d4e-9155-bac5d8f36a30 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:48.703770 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08af7bb-e589-4d4e-9155-bac5d8f36a30 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:48.704232 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a08af7bb-e589-4d4e-9155-bac5d8f36a30 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:48.705228 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a08af7bb-e589-4d4e-9155-bac5d8f36a30 None admin] RBAC: Adding query filter params (interface=public, service_id=892753fc10fb48ceaa7a843eea75e50d, region_id=RegionOne) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 21:59:48.705730 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a08af7bb-e589-4d4e-9155-bac5d8f36a30 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:48.707352 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a08af7bb-e589-4d4e-9155-bac5d8f36a30 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:48.717999 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 388/779] 10.222.0.52 () {56 vars in 1251 bytes} [Wed Jul 13 21:59:48 2022] GET /identity/v3/endpoints?service_id=892753fc10fb48ceaa7a843eea75e50d&interface=public®ion_id=RegionOne => generated 194 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 21:59:48.724114 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95894861-b9f8-46fd-ad19-07a5862d3b6d None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:48.845496 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95894861-b9f8-46fd-ad19-07a5862d3b6d None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:48.882115 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e571a163-583a-4882-8d57-67dc1d6a63b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:48.882863 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e571a163-583a-4882-8d57-67dc1d6a63b2 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:48.883145 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e571a163-583a-4882-8d57-67dc1d6a63b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:48.883424 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e571a163-583a-4882-8d57-67dc1d6a63b2 None admin] PATH_INFO: `/v3/services` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:48.884221 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e571a163-583a-4882-8d57-67dc1d6a63b2 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:48.885722 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e571a163-583a-4882-8d57-67dc1d6a63b2 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:48.891647 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 392/780] 10.222.0.52 () {56 vars in 1087 bytes} [Wed Jul 13 21:59:48 2022] GET /identity/v3/services => generated 772 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 21:59:49.020113 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fbb999-8257-4bcb-9f94-902ee6ef3601 None None] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:49.020395 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fbb999-8257-4bcb-9f94-902ee6ef3601 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:49.020667 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4fbb999-8257-4bcb-9f94-902ee6ef3601 None None] PATH_INFO: `/` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:49.022334 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 389/781] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jul 13 21:59:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 21:59:49.028272 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4542f2d-bd3e-42f6-bed8-fc1098157d72 None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:49.028556 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4542f2d-bd3e-42f6-bed8-fc1098157d72 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:49.028834 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4542f2d-bd3e-42f6-bed8-fc1098157d72 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:49.046766 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-b4542f2d-bd3e-42f6-bed8-fc1098157d72 None None] MFA Rules not processed for user `cc99c93da5074cf5a48bc2aa9a92c5e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:59:49.047964 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-b4542f2d-bd3e-42f6-bed8-fc1098157d72 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:59:49.070785 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:59:49.122005 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 393/782] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jul 13 21:59:49 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 13 21:59:49.144817 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4fbb999-8257-4bcb-9f94-902ee6ef3601 None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:49.161976 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:59:49.270038 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4fbb999-8257-4bcb-9f94-902ee6ef3601 None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:49.275187 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d556fe1-51c6-4ebf-91c7-01c9587dba9d None None] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:49.275474 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d556fe1-51c6-4ebf-91c7-01c9587dba9d None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:49.275749 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d556fe1-51c6-4ebf-91c7-01c9587dba9d None None] PATH_INFO: `/` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:49.277403 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 394/783] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jul 13 21:59:49 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 21:59:49.282665 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22399561-5c57-4c6b-b51c-6afe2ce7bd2c None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:49.282952 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22399561-5c57-4c6b-b51c-6afe2ce7bd2c None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:49.283228 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22399561-5c57-4c6b-b51c-6afe2ce7bd2c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:49.301626 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-22399561-5c57-4c6b-b51c-6afe2ce7bd2c None None] MFA Rules not processed for user `cc99c93da5074cf5a48bc2aa9a92c5e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:59:49.302949 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-22399561-5c57-4c6b-b51c-6afe2ce7bd2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:59:49.320317 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe91350-cfe7-473f-a46f-f9b9ec53d66f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:49.321429 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe91350-cfe7-473f-a46f-f9b9ec53d66f None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:49.321712 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe91350-cfe7-473f-a46f-f9b9ec53d66f None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:49.321987 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fe91350-cfe7-473f-a46f-f9b9ec53d66f None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:49.335078 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fe91350-cfe7-473f-a46f-f9b9ec53d66f None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 21:59:49.343239 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application [None req-0fe91350-cfe7-473f-a46f-f9b9ec53d66f None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 13 21:59:49.343239 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 21:59:49.343239 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 21:59:49.343239 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 21:59:49.343239 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 21:59:49.343239 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 21:59:49.343239 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 21:59:49.343239 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 21:59:49.343239 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 21:59:49.343239 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 21:59:49.343239 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 21:59:49.343239 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 21:59:49.343239 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 21:59:49.343239 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 21:59:49.343239 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 21:59:49.343239 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 21:59:49.344172 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 21:59:49.344172 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 21:59:49.344172 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 21:59:49.344172 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 21:59:49.344172 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 21:59:49.344172 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 21:59:49.344172 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 21:59:49.344172 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 21:59:49.344172 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 21:59:49.344172 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 21:59:49.344172 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 21:59:49.344172 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application with Lock( Jul 13 21:59:49.344172 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 21:59:49.344172 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._enter() Jul 13 21:59:49.344172 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 21:59:49.344172 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 21:59:49.345435 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 21:59:49.345435 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.creator() Jul 13 21:59:49.345435 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 21:59:49.345435 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application created_value = creator( Jul 13 21:59:49.345435 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 21:59:49.345435 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 21:59:49.345435 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 21:59:49.345435 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 21:59:49.345435 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 21:59:49.345435 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 21:59:49.345435 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 13 21:59:49.345435 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Jul 13 21:59:49.345435 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 390/784] 10.222.0.52 () {56 vars in 1096 bytes} [Wed Jul 13 21:59:49 2022] GET /identity/v3/roles/admin => generated 83 bytes in 200 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 21:59:49.350805 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe91350-cfe7-473f-a46f-f9b9ec53d66f None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:49.388546 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 395/785] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jul 13 21:59:49 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 13 21:59:49.423439 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22399561-5c57-4c6b-b51c-6afe2ce7bd2c None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:49.459149 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:59:49.515081 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fe91350-cfe7-473f-a46f-f9b9ec53d66f None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:49.544267 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:59:49.564515 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c94ea3-a47d-4de3-b90c-f26c1c2895b5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:49.565206 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c94ea3-a47d-4de3-b90c-f26c1c2895b5 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:49.565486 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c94ea3-a47d-4de3-b90c-f26c1c2895b5 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:49.565825 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31c94ea3-a47d-4de3-b90c-f26c1c2895b5 None admin] PATH_INFO: `/v3/roles` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:49.566561 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c94ea3-a47d-4de3-b90c-f26c1c2895b5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 21:59:49.566899 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c94ea3-a47d-4de3-b90c-f26c1c2895b5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:49.568339 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31c94ea3-a47d-4de3-b90c-f26c1c2895b5 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:49.573801 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22399561-5c57-4c6b-b51c-6afe2ce7bd2c None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:49.579424 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 391/786] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Jul 13 21:59:49 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 229 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 21:59:49.584901 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c94ea3-a47d-4de3-b90c-f26c1c2895b5 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:49.613738 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50b3a79-c1bc-454c-9ff2-4b7cea4b1e6e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:49.614567 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50b3a79-c1bc-454c-9ff2-4b7cea4b1e6e None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:49.614847 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50b3a79-c1bc-454c-9ff2-4b7cea4b1e6e None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:49.615117 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c50b3a79-c1bc-454c-9ff2-4b7cea4b1e6e None admin] PATH_INFO: `/v3/roles/service` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:49.625681 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c50b3a79-c1bc-454c-9ff2-4b7cea4b1e6e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 21:59:49.633430 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application [None req-c50b3a79-c1bc-454c-9ff2-4b7cea4b1e6e None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 13 21:59:49.633430 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 21:59:49.633430 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 21:59:49.633430 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 21:59:49.633430 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 21:59:49.633430 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 21:59:49.633430 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 21:59:49.633430 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 21:59:49.633430 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 21:59:49.633430 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 21:59:49.633430 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 21:59:49.633430 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 21:59:49.633430 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 21:59:49.633430 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 21:59:49.633430 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 21:59:49.633430 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 21:59:49.634419 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 21:59:49.634419 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 21:59:49.634419 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 21:59:49.634419 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 21:59:49.634419 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 21:59:49.634419 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 21:59:49.634419 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 21:59:49.634419 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 21:59:49.634419 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 21:59:49.634419 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 21:59:49.634419 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 21:59:49.634419 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application with Lock( Jul 13 21:59:49.634419 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 21:59:49.634419 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._enter() Jul 13 21:59:49.634419 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 21:59:49.634419 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 21:59:49.635291 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 21:59:49.635291 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.creator() Jul 13 21:59:49.635291 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 21:59:49.635291 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application created_value = creator( Jul 13 21:59:49.635291 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 21:59:49.635291 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 21:59:49.635291 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 21:59:49.635291 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 21:59:49.635291 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 21:59:49.635291 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 21:59:49.635291 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 13 21:59:49.635291 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Jul 13 21:59:49.635291 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 396/787] 10.222.0.52 () {56 vars in 1102 bytes} [Wed Jul 13 21:59:49 2022] GET /identity/v3/roles/service => generated 85 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 13 21:59:49.640693 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50b3a79-c1bc-454c-9ff2-4b7cea4b1e6e None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:49.778514 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31c94ea3-a47d-4de3-b90c-f26c1c2895b5 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:49.789187 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c50b3a79-c1bc-454c-9ff2-4b7cea4b1e6e None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:49.839348 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef59d1c-b119-4d3f-a1a5-82912a80de6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:49.840159 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef59d1c-b119-4d3f-a1a5-82912a80de6f None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:49.840443 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef59d1c-b119-4d3f-a1a5-82912a80de6f None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:49.840719 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fef59d1c-b119-4d3f-a1a5-82912a80de6f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:49.845090 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:59:49.845464 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd4b0b4-067e-467c-b718-ad2a7c714500 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:49.846152 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd4b0b4-067e-467c-b718-ad2a7c714500 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:49.846437 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd4b0b4-067e-467c-b718-ad2a7c714500 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:49.846741 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfd4b0b4-067e-467c-b718-ad2a7c714500 None admin] PATH_INFO: `/v3/roles` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:49.847496 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd4b0b4-067e-467c-b718-ad2a7c714500 None admin] RBAC: Adding query filter params (name=service) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 21:59:49.847803 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd4b0b4-067e-467c-b718-ad2a7c714500 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:49.849200 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfd4b0b4-067e-467c-b718-ad2a7c714500 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:49.863968 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef59d1c-b119-4d3f-a1a5-82912a80de6f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:49.865431 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fef59d1c-b119-4d3f-a1a5-82912a80de6f None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:49.874579 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 397/788] 10.222.0.52 () {56 vars in 1103 bytes} [Wed Jul 13 21:59:49 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 230 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 21:59:49.879439 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd4b0b4-067e-467c-b718-ad2a7c714500 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:49.893141 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application [None req-fef59d1c-b119-4d3f-a1a5-82912a80de6f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 21:59:49.893141 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 21:59:49.893141 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 21:59:49.893141 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 21:59:49.893141 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 21:59:49.893141 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 21:59:49.893141 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 21:59:49.893141 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 21:59:49.893141 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 21:59:49.893141 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 21:59:49.893141 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 21:59:49.893141 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 21:59:49.893141 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 21:59:49.893141 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 21:59:49.893141 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 21:59:49.894219 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 21:59:49.894219 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 21:59:49.894219 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 21:59:49.894219 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 21:59:49.894219 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 21:59:49.894219 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 21:59:49.894219 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 21:59:49.894219 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 21:59:49.894219 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application with Lock( Jul 13 21:59:49.894219 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 21:59:49.894219 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._enter() Jul 13 21:59:49.894219 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 21:59:49.894219 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 21:59:49.894219 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 21:59:49.894219 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.creator() Jul 13 21:59:49.894219 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 21:59:49.894219 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application created_value = creator( Jul 13 21:59:49.895304 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 21:59:49.895304 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 21:59:49.895304 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 21:59:49.895304 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Jul 13 21:59:49.899811 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 392/789] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Jul 13 21:59:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 315 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 21:59:49.905623 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef59d1c-b119-4d3f-a1a5-82912a80de6f None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:49.907234 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:59:50.055556 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfd4b0b4-067e-467c-b718-ad2a7c714500 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:50.089469 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fef59d1c-b119-4d3f-a1a5-82912a80de6f None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:50.127339 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dda7d05-0d7b-4364-9caa-448c787a8d89 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:50.128313 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dda7d05-0d7b-4364-9caa-448c787a8d89 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:50.128712 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dda7d05-0d7b-4364-9caa-448c787a8d89 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:50.129333 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8dda7d05-0d7b-4364-9caa-448c787a8d89 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:50.153273 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dda7d05-0d7b-4364-9caa-448c787a8d89 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:50.154775 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8dda7d05-0d7b-4364-9caa-448c787a8d89 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:50.155793 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625e3236-4efd-4f90-a6bc-531ac7fc4064 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:50.156467 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625e3236-4efd-4f90-a6bc-531ac7fc4064 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:50.156744 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625e3236-4efd-4f90-a6bc-531ac7fc4064 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:50.157015 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625e3236-4efd-4f90-a6bc-531ac7fc4064 None admin] PATH_INFO: `/v3/domains` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:50.157754 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-625e3236-4efd-4f90-a6bc-531ac7fc4064 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 21:59:50.158064 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-625e3236-4efd-4f90-a6bc-531ac7fc4064 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:50.159445 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-625e3236-4efd-4f90-a6bc-531ac7fc4064 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:50.192474 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 393/790] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Jul 13 21:59:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 292 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 21:59:50.193311 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application [None req-8dda7d05-0d7b-4364-9caa-448c787a8d89 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 21:59:50.193311 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 21:59:50.193311 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 21:59:50.193311 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 21:59:50.193311 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 21:59:50.193311 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 21:59:50.193311 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 21:59:50.193311 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 21:59:50.193311 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 21:59:50.193311 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 21:59:50.193311 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 21:59:50.193311 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 21:59:50.193311 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 21:59:50.193311 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 21:59:50.193311 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 21:59:50.194294 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 21:59:50.194294 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 21:59:50.194294 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 21:59:50.194294 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 21:59:50.194294 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 21:59:50.194294 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 21:59:50.194294 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 21:59:50.194294 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 21:59:50.194294 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application with Lock( Jul 13 21:59:50.194294 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 21:59:50.194294 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._enter() Jul 13 21:59:50.194294 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 21:59:50.194294 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 21:59:50.194294 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 21:59:50.194294 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.creator() Jul 13 21:59:50.194294 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 21:59:50.194294 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application created_value = creator( Jul 13 21:59:50.195230 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 21:59:50.195230 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 21:59:50.195230 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 21:59:50.195230 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Jul 13 21:59:50.195230 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 398/791] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Jul 13 21:59:49 2022] GET /identity/v3/domains/Default => generated 87 bytes in 316 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 13 21:59:50.196738 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625e3236-4efd-4f90-a6bc-531ac7fc4064 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:50.207494 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dda7d05-0d7b-4364-9caa-448c787a8d89 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:50.282513 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:59:50.294925 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:59:50.348335 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8dda7d05-0d7b-4364-9caa-448c787a8d89 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:50.416015 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625e3236-4efd-4f90-a6bc-531ac7fc4064 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:50.421677 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6afecb8-f8ce-4b17-8d30-47df50a82e4e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:50.422621 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6afecb8-f8ce-4b17-8d30-47df50a82e4e None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:50.423069 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6afecb8-f8ce-4b17-8d30-47df50a82e4e None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:50.423483 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6afecb8-f8ce-4b17-8d30-47df50a82e4e None admin] PATH_INFO: `/v3/domains` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:50.425558 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6afecb8-f8ce-4b17-8d30-47df50a82e4e None admin] RBAC: Adding query filter params (name=Default) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 21:59:50.429709 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6afecb8-f8ce-4b17-8d30-47df50a82e4e None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:50.432076 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6afecb8-f8ce-4b17-8d30-47df50a82e4e None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:50.458787 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 399/792] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Jul 13 21:59:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 252 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 21:59:50.461879 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6afecb8-f8ce-4b17-8d30-47df50a82e4e None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:50.493388 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3fe026-b125-4fad-9bb2-db0a98e0d2dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:50.494912 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3fe026-b125-4fad-9bb2-db0a98e0d2dd None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:50.495336 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3fe026-b125-4fad-9bb2-db0a98e0d2dd None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:50.495777 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d3fe026-b125-4fad-9bb2-db0a98e0d2dd None admin] PATH_INFO: `/v3/users/swift` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:50.510479 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3fe026-b125-4fad-9bb2-db0a98e0d2dd None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:50.513764 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d3fe026-b125-4fad-9bb2-db0a98e0d2dd None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:50.536584 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application [None req-4d3fe026-b125-4fad-9bb2-db0a98e0d2dd None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 13 21:59:50.536584 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 21:59:50.536584 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 21:59:50.536584 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 21:59:50.536584 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 21:59:50.536584 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 21:59:50.536584 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 21:59:50.536584 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 21:59:50.536584 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 21:59:50.536584 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 21:59:50.536584 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 21:59:50.536584 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 21:59:50.536584 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 13 21:59:50.536584 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 13 21:59:50.536584 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 13 21:59:50.536584 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 13 21:59:50.537831 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 21:59:50.537831 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 21:59:50.537831 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 13 21:59:50.537831 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 21:59:50.537831 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 13 21:59:50.537831 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 21:59:50.537831 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 21:59:50.537831 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 21:59:50.537831 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 21:59:50.537831 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 21:59:50.537831 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 21:59:50.537831 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application with Lock( Jul 13 21:59:50.537831 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 21:59:50.537831 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._enter() Jul 13 21:59:50.537831 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 21:59:50.537831 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 21:59:50.537831 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 21:59:50.537831 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.creator() Jul 13 21:59:50.539054 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 21:59:50.539054 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application created_value = creator( Jul 13 21:59:50.539054 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 13 21:59:50.539054 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 13 21:59:50.539054 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 13 21:59:50.539054 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 13 21:59:50.539054 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 13 21:59:50.539054 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 13 21:59:50.539054 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 13 21:59:50.539054 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Jul 13 21:59:50.542177 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 394/793] 10.222.0.52 () {56 vars in 1096 bytes} [Wed Jul 13 21:59:50 2022] GET /identity/v3/users/swift => generated 83 bytes in 346 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 13 21:59:50.547928 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3fe026-b125-4fad-9bb2-db0a98e0d2dd None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:50.632710 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6afecb8-f8ce-4b17-8d30-47df50a82e4e None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:50.659473 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d3fe026-b125-4fad-9bb2-db0a98e0d2dd None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:50.668215 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:59:50.669121 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42a26167-1947-4e73-9f31-b5b799e43253 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:50.670179 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a26167-1947-4e73-9f31-b5b799e43253 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:50.670858 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a26167-1947-4e73-9f31-b5b799e43253 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:50.671388 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42a26167-1947-4e73-9f31-b5b799e43253 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:50.674919 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:59:50.682706 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a26167-1947-4e73-9f31-b5b799e43253 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:50.684563 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42a26167-1947-4e73-9f31-b5b799e43253 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:50.700447 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26cb6a54-c6e2-40a0-aa94-ac4cef0ec80f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:50.701127 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cb6a54-c6e2-40a0-aa94-ac4cef0ec80f None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:50.701405 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cb6a54-c6e2-40a0-aa94-ac4cef0ec80f None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:50.701676 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26cb6a54-c6e2-40a0-aa94-ac4cef0ec80f None admin] PATH_INFO: `/v3/users` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:50.702562 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26cb6a54-c6e2-40a0-aa94-ac4cef0ec80f None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 21:59:50.702867 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26cb6a54-c6e2-40a0-aa94-ac4cef0ec80f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:50.704303 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26cb6a54-c6e2-40a0-aa94-ac4cef0ec80f None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:50.706319 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application [None req-42a26167-1947-4e73-9f31-b5b799e43253 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 13 21:59:50.706319 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 21:59:50.706319 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 21:59:50.706319 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 21:59:50.706319 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 21:59:50.706319 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 21:59:50.706319 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 21:59:50.706319 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 21:59:50.706319 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 21:59:50.706319 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 21:59:50.706319 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 21:59:50.706319 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 21:59:50.706319 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 13 21:59:50.706319 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 13 21:59:50.706319 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 13 21:59:50.706319 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 13 21:59:50.707271 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 21:59:50.707271 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 21:59:50.707271 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 13 21:59:50.707271 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 21:59:50.707271 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 13 21:59:50.707271 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 21:59:50.707271 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 21:59:50.707271 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 21:59:50.707271 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 21:59:50.707271 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 21:59:50.707271 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 21:59:50.707271 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application with Lock( Jul 13 21:59:50.707271 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 21:59:50.707271 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._enter() Jul 13 21:59:50.707271 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 21:59:50.707271 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 21:59:50.707271 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 21:59:50.707271 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.creator() Jul 13 21:59:50.708215 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 21:59:50.708215 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application created_value = creator( Jul 13 21:59:50.708215 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 13 21:59:50.708215 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 13 21:59:50.708215 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 13 21:59:50.708215 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 13 21:59:50.708215 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 13 21:59:50.708215 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 13 21:59:50.708215 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 13 21:59:50.708215 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Jul 13 21:59:50.711366 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 400/794] 10.222.0.52 () {56 vars in 1117 bytes} [Wed Jul 13 21:59:50 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 250 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 21:59:50.712867 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a32d6c-8c24-400d-9152-5733459a9aba None None] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:50.713308 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a32d6c-8c24-400d-9152-5733459a9aba None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:50.713719 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48a32d6c-8c24-400d-9152-5733459a9aba None None] PATH_INFO: `/` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:50.720929 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 401/795] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jul 13 21:59:50 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 21:59:50.720929 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a32d6c-8c24-400d-9152-5733459a9aba None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:50.733148 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 395/796] 10.222.0.52 () {56 vars in 1135 bytes} [Wed Jul 13 21:59:50 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 21:59:50.734704 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26c58d6-7508-4429-a6b3-4ea3d69ef2e5 None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:50.735112 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26c58d6-7508-4429-a6b3-4ea3d69ef2e5 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:50.735532 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a26c58d6-7508-4429-a6b3-4ea3d69ef2e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:50.756184 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-a26c58d6-7508-4429-a6b3-4ea3d69ef2e5 None None] MFA Rules not processed for user `cc99c93da5074cf5a48bc2aa9a92c5e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:59:50.757964 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-a26c58d6-7508-4429-a6b3-4ea3d69ef2e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:59:50.842513 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 396/797] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jul 13 21:59:50 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 13 21:59:50.842513 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26c58d6-7508-4429-a6b3-4ea3d69ef2e5 None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:50.867247 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48a32d6c-8c24-400d-9152-5733459a9aba None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:50.913251 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691a9d8b-9820-4982-bee3-b71053964d28 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:50.914376 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691a9d8b-9820-4982-bee3-b71053964d28 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:50.915014 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691a9d8b-9820-4982-bee3-b71053964d28 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:50.915497 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-691a9d8b-9820-4982-bee3-b71053964d28 None admin] PATH_INFO: `/v3/users` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:50.916567 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691a9d8b-9820-4982-bee3-b71053964d28 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 21:59:50.917076 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691a9d8b-9820-4982-bee3-b71053964d28 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:50.918839 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-691a9d8b-9820-4982-bee3-b71053964d28 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:50.938284 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 402/798] 10.222.0.52 () {56 vars in 1149 bytes} [Wed Jul 13 21:59:50 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 220 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 21:59:50.939466 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691a9d8b-9820-4982-bee3-b71053964d28 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:50.940047 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:59:50.948288 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a26c58d6-7508-4429-a6b3-4ea3d69ef2e5 None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:50.984937 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:59:50.989397 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364c22b6-183f-4757-8f21-f5f31da2e6a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:50.990465 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364c22b6-183f-4757-8f21-f5f31da2e6a6 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:50.991044 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364c22b6-183f-4757-8f21-f5f31da2e6a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:50.991554 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-364c22b6-183f-4757-8f21-f5f31da2e6a6 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:51.004623 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364c22b6-183f-4757-8f21-f5f31da2e6a6 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:51.006598 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-364c22b6-183f-4757-8f21-f5f31da2e6a6 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:51.028406 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application [None req-364c22b6-183f-4757-8f21-f5f31da2e6a6 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 21:59:51.028406 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 21:59:51.028406 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 21:59:51.028406 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 21:59:51.028406 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 21:59:51.028406 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 21:59:51.028406 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 21:59:51.028406 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 21:59:51.028406 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 21:59:51.028406 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 21:59:51.028406 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 21:59:51.028406 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 21:59:51.028406 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 21:59:51.028406 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 21:59:51.028406 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 21:59:51.029372 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 21:59:51.029372 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 21:59:51.029372 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 21:59:51.029372 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 21:59:51.029372 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 21:59:51.029372 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 21:59:51.029372 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 21:59:51.029372 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 21:59:51.029372 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application with Lock( Jul 13 21:59:51.029372 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 21:59:51.029372 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._enter() Jul 13 21:59:51.029372 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 21:59:51.029372 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 21:59:51.029372 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 21:59:51.029372 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.creator() Jul 13 21:59:51.029372 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 21:59:51.029372 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application created_value = creator( Jul 13 21:59:51.030256 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 21:59:51.030256 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 21:59:51.030256 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 21:59:51.030256 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Jul 13 21:59:51.031101 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 397/799] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Jul 13 21:59:50 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 21:59:51.032208 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364c22b6-183f-4757-8f21-f5f31da2e6a6 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:51.048630 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-691a9d8b-9820-4982-bee3-b71053964d28 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:51.086587 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c244a28-271c-4dce-a256-9322d845b8d0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:51.087423 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c244a28-271c-4dce-a256-9322d845b8d0 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:51.087692 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c244a28-271c-4dce-a256-9322d845b8d0 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:51.087974 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c244a28-271c-4dce-a256-9322d845b8d0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:51.099167 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c244a28-271c-4dce-a256-9322d845b8d0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 21:59:51.107421 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application [None req-9c244a28-271c-4dce-a256-9322d845b8d0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 13 21:59:51.107421 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 21:59:51.107421 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 21:59:51.107421 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 21:59:51.107421 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 21:59:51.107421 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 21:59:51.107421 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 21:59:51.107421 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 21:59:51.107421 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 21:59:51.107421 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 21:59:51.107421 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 21:59:51.107421 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 21:59:51.107421 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 21:59:51.107421 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 21:59:51.107421 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 21:59:51.107421 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 21:59:51.108638 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 21:59:51.108638 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 21:59:51.108638 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 21:59:51.108638 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 21:59:51.108638 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 21:59:51.108638 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 21:59:51.108638 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 21:59:51.108638 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 21:59:51.108638 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 21:59:51.108638 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 21:59:51.108638 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 21:59:51.108638 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application with Lock( Jul 13 21:59:51.108638 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 21:59:51.108638 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._enter() Jul 13 21:59:51.108638 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 21:59:51.108638 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 21:59:51.109884 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 21:59:51.109884 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.creator() Jul 13 21:59:51.109884 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 21:59:51.109884 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application created_value = creator( Jul 13 21:59:51.109884 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 21:59:51.109884 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 21:59:51.109884 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 21:59:51.109884 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 21:59:51.109884 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 21:59:51.109884 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 21:59:51.109884 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 13 21:59:51.109884 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Jul 13 21:59:51.113660 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 403/800] 10.222.0.52 () {56 vars in 1096 bytes} [Wed Jul 13 21:59:50 2022] GET /identity/v3/roles/admin => generated 83 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 21:59:51.115080 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e6b58c-6a38-4f61-83b6-31205f07758c None None] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:51.115366 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e6b58c-6a38-4f61-83b6-31205f07758c None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:51.115631 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e6b58c-6a38-4f61-83b6-31205f07758c None None] PATH_INFO: `/` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:51.118528 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 404/801] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jul 13 21:59:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 21:59:51.118528 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e6b58c-6a38-4f61-83b6-31205f07758c None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:51.144001 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-364c22b6-183f-4757-8f21-f5f31da2e6a6 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:51.181237 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc3c889-efe0-4efe-a117-9a8a6dd570a1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:51.181937 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc3c889-efe0-4efe-a117-9a8a6dd570a1 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:51.182212 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc3c889-efe0-4efe-a117-9a8a6dd570a1 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:51.182479 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5dc3c889-efe0-4efe-a117-9a8a6dd570a1 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:51.193074 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc3c889-efe0-4efe-a117-9a8a6dd570a1 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:51.194556 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5dc3c889-efe0-4efe-a117-9a8a6dd570a1 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:51.201956 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:59:51.214135 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application [None req-5dc3c889-efe0-4efe-a117-9a8a6dd570a1 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 21:59:51.214135 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 21:59:51.214135 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 21:59:51.214135 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 21:59:51.214135 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 21:59:51.214135 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 21:59:51.214135 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 21:59:51.214135 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 21:59:51.214135 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 21:59:51.214135 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 21:59:51.214135 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 21:59:51.214135 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 21:59:51.214135 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 21:59:51.214135 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 21:59:51.214135 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 21:59:51.215277 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 21:59:51.215277 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 21:59:51.215277 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 21:59:51.215277 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 21:59:51.215277 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 21:59:51.215277 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 21:59:51.215277 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 21:59:51.215277 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 21:59:51.215277 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application with Lock( Jul 13 21:59:51.215277 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 21:59:51.215277 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._enter() Jul 13 21:59:51.215277 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 21:59:51.215277 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 21:59:51.215277 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 21:59:51.215277 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.creator() Jul 13 21:59:51.215277 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 21:59:51.215277 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application created_value = creator( Jul 13 21:59:51.216387 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 21:59:51.216387 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 21:59:51.216387 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 21:59:51.216387 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Jul 13 21:59:51.222175 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 398/802] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Jul 13 21:59:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 21:59:51.222175 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc3c889-efe0-4efe-a117-9a8a6dd570a1 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:51.231433 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:59:51.238241 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e6b58c-6a38-4f61-83b6-31205f07758c None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:51.279489 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f401122-84ac-4049-a4e0-6a98eddb2795 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:51.280156 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f401122-84ac-4049-a4e0-6a98eddb2795 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:51.280407 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f401122-84ac-4049-a4e0-6a98eddb2795 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:51.280697 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f401122-84ac-4049-a4e0-6a98eddb2795 None admin] PATH_INFO: `/v3/domains` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:51.281412 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f401122-84ac-4049-a4e0-6a98eddb2795 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 21:59:51.281796 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f401122-84ac-4049-a4e0-6a98eddb2795 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:51.283141 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f401122-84ac-4049-a4e0-6a98eddb2795 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:51.295207 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 405/803] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Jul 13 21:59:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 21:59:51.296843 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9709cf6-fbcc-4cf9-911c-c801fe19c59f None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:51.297128 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9709cf6-fbcc-4cf9-911c-c801fe19c59f None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:51.297408 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9709cf6-fbcc-4cf9-911c-c801fe19c59f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:51.314444 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-f9709cf6-fbcc-4cf9-911c-c801fe19c59f None None] MFA Rules not processed for user `cc99c93da5074cf5a48bc2aa9a92c5e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:59:51.315663 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-f9709cf6-fbcc-4cf9-911c-c801fe19c59f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:59:51.333656 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5dc3c889-efe0-4efe-a117-9a8a6dd570a1 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:51.370656 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1cf21df-49f6-4eec-9fff-837cf6feb3f5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:51.371330 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cf21df-49f6-4eec-9fff-837cf6feb3f5 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:51.371608 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cf21df-49f6-4eec-9fff-837cf6feb3f5 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:51.371879 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1cf21df-49f6-4eec-9fff-837cf6feb3f5 None admin] PATH_INFO: `/v3/roles` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:51.372634 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1cf21df-49f6-4eec-9fff-837cf6feb3f5 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 21:59:51.372941 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1cf21df-49f6-4eec-9fff-837cf6feb3f5 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:51.374320 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1cf21df-49f6-4eec-9fff-837cf6feb3f5 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:51.387645 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 399/804] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Jul 13 21:59:51 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 21:59:51.387645 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1cf21df-49f6-4eec-9fff-837cf6feb3f5 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:51.394635 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 406/805] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jul 13 21:59:51 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 13 21:59:51.396129 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9709cf6-fbcc-4cf9-911c-c801fe19c59f None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:51.451109 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:59:51.492572 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:59:51.492776 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1cf21df-49f6-4eec-9fff-837cf6feb3f5 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:51.504415 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9709cf6-fbcc-4cf9-911c-c801fe19c59f None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:51.525951 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e32883b-57bc-4fc7-9e40-a9686dfedd84 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:51.526644 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e32883b-57bc-4fc7-9e40-a9686dfedd84 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:51.526919 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e32883b-57bc-4fc7-9e40-a9686dfedd84 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:51.527184 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e32883b-57bc-4fc7-9e40-a9686dfedd84 None admin] PATH_INFO: `/v3/domains` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:51.527903 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e32883b-57bc-4fc7-9e40-a9686dfedd84 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 21:59:51.528200 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e32883b-57bc-4fc7-9e40-a9686dfedd84 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:51.529571 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e32883b-57bc-4fc7-9e40-a9686dfedd84 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:51.540669 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee70db2-dcdd-480f-8a56-3f0ca2ffd45a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:51.541482 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee70db2-dcdd-480f-8a56-3f0ca2ffd45a None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:51.541757 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee70db2-dcdd-480f-8a56-3f0ca2ffd45a None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:51.541976 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 400/806] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Jul 13 21:59:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 21:59:51.542084 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ee70db2-dcdd-480f-8a56-3f0ca2ffd45a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:51.542806 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e32883b-57bc-4fc7-9e40-a9686dfedd84 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:51.549927 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee70db2-dcdd-480f-8a56-3f0ca2ffd45a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:51.551478 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ee70db2-dcdd-480f-8a56-3f0ca2ffd45a None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:51.566243 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application [None req-6ee70db2-dcdd-480f-8a56-3f0ca2ffd45a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 13 21:59:51.566243 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 21:59:51.566243 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 21:59:51.566243 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 21:59:51.566243 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 21:59:51.566243 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 21:59:51.566243 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 21:59:51.566243 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 21:59:51.566243 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 21:59:51.566243 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 21:59:51.566243 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 21:59:51.566243 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 21:59:51.566243 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 13 21:59:51.566243 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 13 21:59:51.566243 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 13 21:59:51.567184 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 13 21:59:51.567184 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 21:59:51.567184 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 21:59:51.567184 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 21:59:51.567184 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 21:59:51.567184 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 21:59:51.567184 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 21:59:51.567184 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 21:59:51.567184 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application with Lock( Jul 13 21:59:51.567184 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 21:59:51.567184 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._enter() Jul 13 21:59:51.567184 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 21:59:51.567184 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 21:59:51.567184 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 21:59:51.567184 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.creator() Jul 13 21:59:51.567184 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 21:59:51.567184 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application created_value = creator( Jul 13 21:59:51.568550 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 13 21:59:51.568550 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 13 21:59:51.568550 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 13 21:59:51.568550 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 13 21:59:51.568550 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 13 21:59:51.568550 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 13 21:59:51.568550 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 13 21:59:51.568550 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Jul 13 21:59:51.568550 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 407/807] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Jul 13 21:59:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 21:59:51.568550 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee70db2-dcdd-480f-8a56-3f0ca2ffd45a None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:51.650710 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e32883b-57bc-4fc7-9e40-a9686dfedd84 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:51.670721 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:59:51.675741 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ee70db2-dcdd-480f-8a56-3f0ca2ffd45a None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:51.685379 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0144baec-7a9f-4dee-8b44-4a345df2dd70 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:51.686179 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0144baec-7a9f-4dee-8b44-4a345df2dd70 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:51.686467 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0144baec-7a9f-4dee-8b44-4a345df2dd70 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:51.686780 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0144baec-7a9f-4dee-8b44-4a345df2dd70 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:51.696724 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0144baec-7a9f-4dee-8b44-4a345df2dd70 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:51.698171 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0144baec-7a9f-4dee-8b44-4a345df2dd70 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:51.709344 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a878ae-5553-449a-a244-d5b15873a1c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:51.710097 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a878ae-5553-449a-a244-d5b15873a1c8 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:51.710374 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a878ae-5553-449a-a244-d5b15873a1c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:51.710674 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8a878ae-5553-449a-a244-d5b15873a1c8 None admin] PATH_INFO: `/v3/services/block-storage` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:51.711333 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a878ae-5553-449a-a244-d5b15873a1c8 None admin] RBAC: Authorizing `identity:get_service(service_id=block-storage)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:51.713274 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8a878ae-5553-449a-a244-d5b15873a1c8 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:51.715677 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application [None req-0144baec-7a9f-4dee-8b44-4a345df2dd70 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 21:59:51.715677 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 21:59:51.715677 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 21:59:51.715677 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 21:59:51.715677 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 21:59:51.715677 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 21:59:51.715677 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 21:59:51.715677 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 21:59:51.715677 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 21:59:51.715677 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 21:59:51.715677 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 21:59:51.715677 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 21:59:51.715677 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 21:59:51.715677 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 21:59:51.715677 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 21:59:51.716749 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 21:59:51.716749 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 21:59:51.716749 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 21:59:51.716749 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 21:59:51.716749 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 21:59:51.716749 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 21:59:51.716749 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 21:59:51.716749 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 21:59:51.716749 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application with Lock( Jul 13 21:59:51.716749 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 21:59:51.716749 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._enter() Jul 13 21:59:51.716749 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 21:59:51.716749 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 21:59:51.716749 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 21:59:51.716749 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.creator() Jul 13 21:59:51.716749 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 21:59:51.716749 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application created_value = creator( Jul 13 21:59:51.717840 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 21:59:51.717840 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 21:59:51.717840 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 21:59:51.717840 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Jul 13 21:59:51.717840 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 401/808] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Jul 13 21:59:51 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 21:59:51.718174 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a229097d-4ca4-4625-9999-851f3e27b2e4 None None] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:51.718328 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a229097d-4ca4-4625-9999-851f3e27b2e4 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:51.718633 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a229097d-4ca4-4625-9999-851f3e27b2e4 None None] PATH_INFO: `/` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:51.719921 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 402/809] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jul 13 21:59:51 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 21:59:51.720601 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a229097d-4ca4-4625-9999-851f3e27b2e4 None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:51.732201 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application [None req-d8a878ae-5553-449a-a244-d5b15873a1c8 None admin] Could not find service: block-storage.: keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 13 21:59:51.732201 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 21:59:51.732201 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 13 21:59:51.732201 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 13 21:59:51.732201 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 13 21:59:51.732201 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 13 21:59:51.732201 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 13 21:59:51.732201 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 13 21:59:51.732201 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 13 21:59:51.732201 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Jul 13 21:59:51.732201 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 13 21:59:51.732201 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Jul 13 21:59:51.732201 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 21:59:51.732201 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 21:59:51.732201 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 21:59:51.732201 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 21:59:51.733314 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 21:59:51.733314 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 21:59:51.733314 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 21:59:51.733314 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 21:59:51.733314 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 21:59:51.733314 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 21:59:51.733314 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 21:59:51.733314 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 13 21:59:51.733314 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 13 21:59:51.733314 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 13 21:59:51.733314 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 13 21:59:51.733314 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 21:59:51.733314 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 21:59:51.733314 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 21:59:51.733314 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 21:59:51.733314 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 21:59:51.734358 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 21:59:51.734358 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 21:59:51.734358 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application with Lock( Jul 13 21:59:51.734358 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 21:59:51.734358 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._enter() Jul 13 21:59:51.734358 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 21:59:51.734358 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 21:59:51.734358 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 21:59:51.734358 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.creator() Jul 13 21:59:51.734358 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 21:59:51.734358 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application created_value = creator( Jul 13 21:59:51.734358 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 13 21:59:51.734358 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 13 21:59:51.734358 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: block-storage. Jul 13 21:59:51.734358 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Jul 13 21:59:51.734358 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 408/810] 10.222.0.52 () {56 vars in 1129 bytes} [Wed Jul 13 21:59:51 2022] GET /identity/v3/services/block-storage => generated 94 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 21:59:51.735499 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a878ae-5553-449a-a244-d5b15873a1c8 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:51.750435 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:59:51.827998 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a229097d-4ca4-4625-9999-851f3e27b2e4 None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:51.836713 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8a878ae-5553-449a-a244-d5b15873a1c8 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:51.863180 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1284376-b11b-4faf-a56d-348880d1544c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:51.863984 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1284376-b11b-4faf-a56d-348880d1544c None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:51.864262 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1284376-b11b-4faf-a56d-348880d1544c None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:51.864533 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1284376-b11b-4faf-a56d-348880d1544c None admin] PATH_INFO: `/v3/projects/service` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:51.871608 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3589bbe3-d8e6-44b4-ad99-09982b68d3a4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:51.871764 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1284376-b11b-4faf-a56d-348880d1544c None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:51.872276 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3589bbe3-d8e6-44b4-ad99-09982b68d3a4 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:51.872551 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3589bbe3-d8e6-44b4-ad99-09982b68d3a4 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:51.872825 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3589bbe3-d8e6-44b4-ad99-09982b68d3a4 None admin] PATH_INFO: `/v3/projects` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:51.873117 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1284376-b11b-4faf-a56d-348880d1544c None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:51.873618 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3589bbe3-d8e6-44b4-ad99-09982b68d3a4 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 21:59:51.873927 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3589bbe3-d8e6-44b4-ad99-09982b68d3a4 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:51.875374 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3589bbe3-d8e6-44b4-ad99-09982b68d3a4 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:51.886851 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application [None req-b1284376-b11b-4faf-a56d-348880d1544c None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 13 21:59:51.886851 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 21:59:51.886851 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 21:59:51.886851 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 21:59:51.886851 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 21:59:51.886851 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 21:59:51.886851 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 21:59:51.886851 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 21:59:51.886851 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 21:59:51.886851 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 21:59:51.886851 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 21:59:51.886851 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 21:59:51.886851 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 13 21:59:51.886851 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 13 21:59:51.886851 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 13 21:59:51.887806 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 13 21:59:51.887806 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 21:59:51.887806 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 21:59:51.887806 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 21:59:51.887806 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 21:59:51.887806 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 21:59:51.887806 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 21:59:51.887806 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 21:59:51.887806 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application with Lock( Jul 13 21:59:51.887806 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 21:59:51.887806 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._enter() Jul 13 21:59:51.887806 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 21:59:51.887806 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 21:59:51.887806 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 21:59:51.887806 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.creator() Jul 13 21:59:51.887806 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 21:59:51.887806 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application created_value = creator( Jul 13 21:59:51.888695 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 13 21:59:51.888695 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 13 21:59:51.888695 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 13 21:59:51.888695 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 13 21:59:51.888695 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 13 21:59:51.888695 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 13 21:59:51.888695 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 13 21:59:51.888695 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Jul 13 21:59:51.888695 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 403/811] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Jul 13 21:59:51 2022] GET /identity/v3/projects/service => generated 88 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 21:59:51.889179 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 409/812] 10.222.0.52 () {56 vars in 1148 bytes} [Wed Jul 13 21:59:51 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 21:59:51.889250 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1284376-b11b-4faf-a56d-348880d1544c None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:51.889829 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85a12d1-424c-411f-9b8f-a609ea50d555 None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:51.890110 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85a12d1-424c-411f-9b8f-a609ea50d555 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:51.890388 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a85a12d1-424c-411f-9b8f-a609ea50d555 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:51.909715 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-a85a12d1-424c-411f-9b8f-a609ea50d555 None None] MFA Rules not processed for user `cc99c93da5074cf5a48bc2aa9a92c5e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:59:51.911281 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-a85a12d1-424c-411f-9b8f-a609ea50d555 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:59:51.926923 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:59:51.976266 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 410/813] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jul 13 21:59:51 2022] POST /identity/v3/auth/tokens => generated 1002 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 13 21:59:51.977011 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85a12d1-424c-411f-9b8f-a609ea50d555 None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:51.979952 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:59:51.996040 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1284376-b11b-4faf-a56d-348880d1544c None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:52.029516 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c23a6a8-67dc-490d-be21-94b2a7fb826c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:52.030184 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c23a6a8-67dc-490d-be21-94b2a7fb826c None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:52.030466 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c23a6a8-67dc-490d-be21-94b2a7fb826c None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:52.030767 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c23a6a8-67dc-490d-be21-94b2a7fb826c None admin] PATH_INFO: `/v3/domains` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:52.031489 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c23a6a8-67dc-490d-be21-94b2a7fb826c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 21:59:52.031792 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c23a6a8-67dc-490d-be21-94b2a7fb826c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:52.033180 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c23a6a8-67dc-490d-be21-94b2a7fb826c None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:52.045860 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 404/814] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Jul 13 21:59:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 21:59:52.046454 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c23a6a8-67dc-490d-be21-94b2a7fb826c None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:52.084824 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a85a12d1-424c-411f-9b8f-a609ea50d555 None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:52.118208 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d94c0d-f011-4274-9584-bc26b801d375 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:52.118921 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d94c0d-f011-4274-9584-bc26b801d375 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:52.119201 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d94c0d-f011-4274-9584-bc26b801d375 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:52.119473 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5d94c0d-f011-4274-9584-bc26b801d375 None admin] PATH_INFO: `/v3/services` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:52.120188 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d94c0d-f011-4274-9584-bc26b801d375 None admin] RBAC: Adding query filter params (name=block-storage) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 21:59:52.120491 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d94c0d-f011-4274-9584-bc26b801d375 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:52.121871 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5d94c0d-f011-4274-9584-bc26b801d375 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:52.126363 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 411/815] 10.222.0.52 () {56 vars in 1124 bytes} [Wed Jul 13 21:59:51 2022] GET /identity/v3/services?name=block-storage => generated 130 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 21:59:52.127138 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d94c0d-f011-4274-9584-bc26b801d375 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:52.138220 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:59:52.147943 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c23a6a8-67dc-490d-be21-94b2a7fb826c None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:52.183455 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7e0c72-73e2-4983-8bdc-87238b51ef13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:52.184092 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7e0c72-73e2-4983-8bdc-87238b51ef13 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:52.184366 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7e0c72-73e2-4983-8bdc-87238b51ef13 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:52.184637 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce7e0c72-73e2-4983-8bdc-87238b51ef13 None admin] PATH_INFO: `/v3/projects` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:52.185492 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7e0c72-73e2-4983-8bdc-87238b51ef13 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 21:59:52.185794 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7e0c72-73e2-4983-8bdc-87238b51ef13 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:52.187291 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce7e0c72-73e2-4983-8bdc-87238b51ef13 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:52.192739 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:59:52.199399 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 405/816] 10.222.0.52 () {56 vars in 1148 bytes} [Wed Jul 13 21:59:52 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 21:59:52.200176 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7e0c72-73e2-4983-8bdc-87238b51ef13 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:52.227200 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5d94c0d-f011-4274-9584-bc26b801d375 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:52.262205 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2818504-06bc-4e19-8b73-562ad60f33eb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:52.263136 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2818504-06bc-4e19-8b73-562ad60f33eb None admin] REQUEST_METHOD: `PUT` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:52.263414 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2818504-06bc-4e19-8b73-562ad60f33eb None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:52.263690 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2818504-06bc-4e19-8b73-562ad60f33eb None admin] PATH_INFO: `/v3/projects/417e3e2f1333481795a089bef2e24bea/users/1eb1525233ba48c99b15d4cb75db4822/roles/44efba2579f342df8a0789c59d163ec3` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:52.265740 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2818504-06bc-4e19-8b73-562ad60f33eb None admin] RBAC: Authorizing `identity:create_grant(project_id=417e3e2f1333481795a089bef2e24bea, user_id=1eb1525233ba48c99b15d4cb75db4822, role_id=44efba2579f342df8a0789c59d163ec3)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:52.267252 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2818504-06bc-4e19-8b73-562ad60f33eb None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:52.281748 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 412/817] 10.222.0.52 () {58 vars in 1439 bytes} [Wed Jul 13 21:59:52 2022] PUT /identity/v3/projects/417e3e2f1333481795a089bef2e24bea/users/1eb1525233ba48c99b15d4cb75db4822/roles/44efba2579f342df8a0789c59d163ec3 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 21:59:52.282575 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2818504-06bc-4e19-8b73-562ad60f33eb None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:52.313382 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce7e0c72-73e2-4983-8bdc-87238b51ef13 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:52.355313 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b93e30b-5949-4109-872d-6393d333f3ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:52.356290 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b93e30b-5949-4109-872d-6393d333f3ca None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:52.356574 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b93e30b-5949-4109-872d-6393d333f3ca None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:52.356854 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b93e30b-5949-4109-872d-6393d333f3ca None admin] PATH_INFO: `/v3/services/network` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:52.357528 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b93e30b-5949-4109-872d-6393d333f3ca None admin] RBAC: Authorizing `identity:get_service(service_id=network)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:52.359130 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b93e30b-5949-4109-872d-6393d333f3ca None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:52.372624 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application [None req-9b93e30b-5949-4109-872d-6393d333f3ca None admin] Could not find service: network.: keystone.exception.ServiceNotFound: Could not find service: network. Jul 13 21:59:52.372624 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 21:59:52.372624 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 13 21:59:52.372624 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 13 21:59:52.372624 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 13 21:59:52.372624 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 13 21:59:52.372624 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 13 21:59:52.372624 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 13 21:59:52.372624 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 13 21:59:52.372624 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Jul 13 21:59:52.372624 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 13 21:59:52.372624 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Jul 13 21:59:52.372624 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 21:59:52.372624 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 21:59:52.372624 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 21:59:52.372624 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 21:59:52.372624 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 21:59:52.373698 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 21:59:52.373698 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 21:59:52.373698 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 21:59:52.373698 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 21:59:52.373698 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 21:59:52.373698 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 21:59:52.373698 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 13 21:59:52.373698 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 13 21:59:52.373698 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 13 21:59:52.373698 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 13 21:59:52.373698 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 21:59:52.373698 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 21:59:52.373698 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 21:59:52.373698 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 21:59:52.373698 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 21:59:52.373698 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 21:59:52.374558 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 21:59:52.374558 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application with Lock( Jul 13 21:59:52.374558 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 21:59:52.374558 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._enter() Jul 13 21:59:52.374558 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 21:59:52.374558 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 21:59:52.374558 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 21:59:52.374558 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.creator() Jul 13 21:59:52.374558 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 21:59:52.374558 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application created_value = creator( Jul 13 21:59:52.374558 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 13 21:59:52.374558 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 13 21:59:52.374558 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: network. Jul 13 21:59:52.374558 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Jul 13 21:59:52.374558 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 406/818] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Jul 13 21:59:52 2022] GET /identity/v3/services/network => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 21:59:52.375379 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b93e30b-5949-4109-872d-6393d333f3ca None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:52.400629 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:59:52.402400 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2818504-06bc-4e19-8b73-562ad60f33eb None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:52.431706 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:59:52.443343 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d0ad6a-44ca-422d-9e59-6df6e85b8927 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:52.444189 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d0ad6a-44ca-422d-9e59-6df6e85b8927 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:52.445000 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d0ad6a-44ca-422d-9e59-6df6e85b8927 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:52.445907 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14d0ad6a-44ca-422d-9e59-6df6e85b8927 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:52.456826 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d0ad6a-44ca-422d-9e59-6df6e85b8927 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:52.458268 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14d0ad6a-44ca-422d-9e59-6df6e85b8927 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:52.472403 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application [None req-14d0ad6a-44ca-422d-9e59-6df6e85b8927 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 13 21:59:52.472403 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 21:59:52.472403 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 21:59:52.472403 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 21:59:52.472403 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 21:59:52.472403 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 21:59:52.472403 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 21:59:52.472403 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 21:59:52.472403 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 21:59:52.472403 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 21:59:52.472403 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 21:59:52.472403 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 21:59:52.472403 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 13 21:59:52.472403 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 13 21:59:52.472403 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 13 21:59:52.472403 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 13 21:59:52.473462 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 21:59:52.473462 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 21:59:52.473462 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 13 21:59:52.473462 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 21:59:52.473462 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 13 21:59:52.473462 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 21:59:52.473462 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 21:59:52.473462 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 21:59:52.473462 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 21:59:52.473462 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 21:59:52.473462 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 21:59:52.473462 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application with Lock( Jul 13 21:59:52.473462 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 21:59:52.473462 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._enter() Jul 13 21:59:52.473462 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 21:59:52.473462 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 21:59:52.473462 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 21:59:52.473462 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.creator() Jul 13 21:59:52.474403 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 21:59:52.474403 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application created_value = creator( Jul 13 21:59:52.474403 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 13 21:59:52.474403 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 13 21:59:52.474403 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 13 21:59:52.474403 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 13 21:59:52.474403 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 13 21:59:52.474403 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 13 21:59:52.474403 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 13 21:59:52.474403 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Jul 13 21:59:52.474403 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 413/819] 10.222.0.52 () {56 vars in 1093 bytes} [Wed Jul 13 21:59:52 2022] GET /identity/v3/users/nova => generated 82 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 21:59:52.474403 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d0ad6a-44ca-422d-9e59-6df6e85b8927 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:52.494925 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b93e30b-5949-4109-872d-6393d333f3ca None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:52.530403 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a08485-baf1-49bf-94c9-d5804ed3aa60 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:52.531099 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a08485-baf1-49bf-94c9-d5804ed3aa60 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:52.531375 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a08485-baf1-49bf-94c9-d5804ed3aa60 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:52.531645 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63a08485-baf1-49bf-94c9-d5804ed3aa60 None admin] PATH_INFO: `/v3/services` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:52.532466 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a08485-baf1-49bf-94c9-d5804ed3aa60 None admin] RBAC: Adding query filter params (type=block-storage) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 21:59:52.532774 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a08485-baf1-49bf-94c9-d5804ed3aa60 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:52.534146 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63a08485-baf1-49bf-94c9-d5804ed3aa60 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:52.542245 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 407/820] 10.222.0.52 () {56 vars in 1124 bytes} [Wed Jul 13 21:59:52 2022] GET /identity/v3/services?type=block-storage => generated 367 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 21:59:52.543100 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a08485-baf1-49bf-94c9-d5804ed3aa60 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:52.584955 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14d0ad6a-44ca-422d-9e59-6df6e85b8927 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:52.620151 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6f39bd-f069-462d-8b1b-27c3cb4d0ae0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:52.621046 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6f39bd-f069-462d-8b1b-27c3cb4d0ae0 None admin] REQUEST_METHOD: `PUT` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:52.621320 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6f39bd-f069-462d-8b1b-27c3cb4d0ae0 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:52.621593 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d6f39bd-f069-462d-8b1b-27c3cb4d0ae0 None admin] PATH_INFO: `/v3/projects/417e3e2f1333481795a089bef2e24bea/users/b866bacdafae4db6a32714c07c56515f/roles/70c34ea493484322911dab62b4a56e75` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:52.628816 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:59:52.637940 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6f39bd-f069-462d-8b1b-27c3cb4d0ae0 None admin] RBAC: Authorizing `identity:create_grant(project_id=417e3e2f1333481795a089bef2e24bea, user_id=b866bacdafae4db6a32714c07c56515f, role_id=70c34ea493484322911dab62b4a56e75)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:52.639422 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d6f39bd-f069-462d-8b1b-27c3cb4d0ae0 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:52.651115 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63a08485-baf1-49bf-94c9-d5804ed3aa60 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:52.662519 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 414/821] 10.222.0.52 () {58 vars in 1439 bytes} [Wed Jul 13 21:59:52 2022] PUT /identity/v3/projects/417e3e2f1333481795a089bef2e24bea/users/b866bacdafae4db6a32714c07c56515f/roles/70c34ea493484322911dab62b4a56e75 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 21:59:52.662519 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6f39bd-f069-462d-8b1b-27c3cb4d0ae0 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:52.699870 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1817236c-6ea9-4dc9-b62d-6b2637a0cd4a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:52.700705 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1817236c-6ea9-4dc9-b62d-6b2637a0cd4a None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:52.701046 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1817236c-6ea9-4dc9-b62d-6b2637a0cd4a None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:52.701433 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1817236c-6ea9-4dc9-b62d-6b2637a0cd4a None admin] PATH_INFO: `/v3/services` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:52.702243 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1817236c-6ea9-4dc9-b62d-6b2637a0cd4a None admin] RBAC: Adding query filter params (name=network) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 21:59:52.702651 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1817236c-6ea9-4dc9-b62d-6b2637a0cd4a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:52.704151 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1817236c-6ea9-4dc9-b62d-6b2637a0cd4a None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:52.709900 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 408/822] 10.222.0.52 () {56 vars in 1112 bytes} [Wed Jul 13 21:59:52 2022] GET /identity/v3/services?name=network => generated 124 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 21:59:52.710992 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1817236c-6ea9-4dc9-b62d-6b2637a0cd4a None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:52.715469 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:59:52.790607 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d6f39bd-f069-462d-8b1b-27c3cb4d0ae0 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:52.835717 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0e622e-8454-4cb4-9d06-02023aab6758 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:52.836708 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0e622e-8454-4cb4-9d06-02023aab6758 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:52.837052 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0e622e-8454-4cb4-9d06-02023aab6758 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:52.837613 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba0e622e-8454-4cb4-9d06-02023aab6758 None admin] PATH_INFO: `/v3/users` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:52.838747 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba0e622e-8454-4cb4-9d06-02023aab6758 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 21:59:52.839064 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba0e622e-8454-4cb4-9d06-02023aab6758 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:52.841251 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba0e622e-8454-4cb4-9d06-02023aab6758 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:52.849186 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1817236c-6ea9-4dc9-b62d-6b2637a0cd4a None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:52.868972 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 415/823] 10.222.0.52 () {56 vars in 1133 bytes} [Wed Jul 13 21:59:52 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 215 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 21:59:52.869822 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0e622e-8454-4cb4-9d06-02023aab6758 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:52.889767 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f24fb7-9b92-499e-a6ec-12522f003960 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:52.890441 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f24fb7-9b92-499e-a6ec-12522f003960 None admin] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:52.890760 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f24fb7-9b92-499e-a6ec-12522f003960 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:52.891038 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66f24fb7-9b92-499e-a6ec-12522f003960 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:52.893206 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f24fb7-9b92-499e-a6ec-12522f003960 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:52.893206 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66f24fb7-9b92-499e-a6ec-12522f003960 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:52.922710 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:59:52.931612 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 409/824] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Jul 13 21:59:52 2022] POST /identity/v3/endpoints => generated 346 bytes in 221 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 21:59:52.932592 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f24fb7-9b92-499e-a6ec-12522f003960 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:52.982966 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:59:52.994874 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba0e622e-8454-4cb4-9d06-02023aab6758 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:53.039623 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d631b4-5f0d-4041-8d12-eb3a21f904dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:53.040394 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d631b4-5f0d-4041-8d12-eb3a21f904dd None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:53.040784 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d631b4-5f0d-4041-8d12-eb3a21f904dd None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:53.040989 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98d631b4-5f0d-4041-8d12-eb3a21f904dd None admin] PATH_INFO: `/v3/services` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:53.041715 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d631b4-5f0d-4041-8d12-eb3a21f904dd None admin] RBAC: Adding query filter params (type=network) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 21:59:53.042055 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d631b4-5f0d-4041-8d12-eb3a21f904dd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:53.043561 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98d631b4-5f0d-4041-8d12-eb3a21f904dd None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:53.049626 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 416/825] 10.222.0.52 () {56 vars in 1112 bytes} [Wed Jul 13 21:59:52 2022] GET /identity/v3/services?type=network => generated 350 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 21:59:53.055541 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d631b4-5f0d-4041-8d12-eb3a21f904dd None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:53.075713 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66f24fb7-9b92-499e-a6ec-12522f003960 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:53.163243 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07198d48-99d8-4081-a063-584b0e256ff1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:53.163912 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07198d48-99d8-4081-a063-584b0e256ff1 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:53.164190 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07198d48-99d8-4081-a063-584b0e256ff1 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:53.164457 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07198d48-99d8-4081-a063-584b0e256ff1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:53.213603 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98d631b4-5f0d-4041-8d12-eb3a21f904dd None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:53.244500 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:59:53.247619 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07198d48-99d8-4081-a063-584b0e256ff1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:53.249145 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07198d48-99d8-4081-a063-584b0e256ff1 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:53.257011 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8538dd48-4ff6-4ada-9810-ae184496ef37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:53.257932 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8538dd48-4ff6-4ada-9810-ae184496ef37 None admin] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:53.258863 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8538dd48-4ff6-4ada-9810-ae184496ef37 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:53.259329 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8538dd48-4ff6-4ada-9810-ae184496ef37 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:53.260250 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8538dd48-4ff6-4ada-9810-ae184496ef37 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:53.262177 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8538dd48-4ff6-4ada-9810-ae184496ef37 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:53.291475 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 417/826] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Jul 13 21:59:53 2022] POST /identity/v3/endpoints => generated 327 bytes in 236 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 13 21:59:53.386035 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 410/827] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Jul 13 21:59:52 2022] GET /identity/v3/auth/tokens => generated 1002 bytes in 454 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 13 21:59:53.388306 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8538dd48-4ff6-4ada-9810-ae184496ef37 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:53.406422 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:59:53.523233 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8538dd48-4ff6-4ada-9810-ae184496ef37 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:53.561048 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044247e7-c484-4bd1-81ea-829b1dfe0f10 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:53.562145 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044247e7-c484-4bd1-81ea-829b1dfe0f10 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:53.562695 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044247e7-c484-4bd1-81ea-829b1dfe0f10 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:53.563143 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-044247e7-c484-4bd1-81ea-829b1dfe0f10 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:53.575409 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044247e7-c484-4bd1-81ea-829b1dfe0f10 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:53.577134 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-044247e7-c484-4bd1-81ea-829b1dfe0f10 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:53.595999 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application [None req-044247e7-c484-4bd1-81ea-829b1dfe0f10 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 21:59:53.595999 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 21:59:53.595999 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 21:59:53.595999 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 21:59:53.595999 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 21:59:53.595999 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 21:59:53.595999 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 21:59:53.595999 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 21:59:53.595999 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 21:59:53.595999 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 21:59:53.595999 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 21:59:53.595999 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 21:59:53.595999 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 21:59:53.595999 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 21:59:53.595999 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 21:59:53.596949 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 21:59:53.596949 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 21:59:53.596949 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 21:59:53.596949 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 21:59:53.596949 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 21:59:53.596949 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 21:59:53.596949 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 21:59:53.596949 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 21:59:53.596949 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application with Lock( Jul 13 21:59:53.596949 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 21:59:53.596949 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._enter() Jul 13 21:59:53.596949 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 21:59:53.596949 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 21:59:53.596949 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 21:59:53.596949 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.creator() Jul 13 21:59:53.596949 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 21:59:53.596949 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application created_value = creator( Jul 13 21:59:53.597837 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 21:59:53.597837 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 21:59:53.597837 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 21:59:53.597837 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Jul 13 21:59:53.599941 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 418/828] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Jul 13 21:59:53 2022] GET /identity/v3/domains/Default => generated 87 bytes in 212 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 21:59:53.606126 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07198d48-99d8-4081-a063-584b0e256ff1 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:53.704940 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:59:53.711391 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07198d48-99d8-4081-a063-584b0e256ff1 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:53.758412 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5862ab0f-f677-4f2f-860d-f62ccc19c446 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:53.759154 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5862ab0f-f677-4f2f-860d-f62ccc19c446 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:53.759429 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5862ab0f-f677-4f2f-860d-f62ccc19c446 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:53.759703 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5862ab0f-f677-4f2f-860d-f62ccc19c446 None admin] PATH_INFO: `/v3/domains` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:53.760438 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5862ab0f-f677-4f2f-860d-f62ccc19c446 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 21:59:53.760739 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5862ab0f-f677-4f2f-860d-f62ccc19c446 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:53.762105 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5862ab0f-f677-4f2f-860d-f62ccc19c446 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:53.778844 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 411/829] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Jul 13 21:59:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 21:59:53.780285 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044247e7-c484-4bd1-81ea-829b1dfe0f10 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:53.845858 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:59:53.890464 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-044247e7-c484-4bd1-81ea-829b1dfe0f10 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:53.925016 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2de869-d7a3-4cb8-813c-780a90aba054 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:53.925823 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2de869-d7a3-4cb8-813c-780a90aba054 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:53.926097 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2de869-d7a3-4cb8-813c-780a90aba054 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:53.926366 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2de869-d7a3-4cb8-813c-780a90aba054 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:53.933420 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2de869-d7a3-4cb8-813c-780a90aba054 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:53.934901 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2de869-d7a3-4cb8-813c-780a90aba054 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:53.949754 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application [None req-8d2de869-d7a3-4cb8-813c-780a90aba054 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 13 21:59:53.949754 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 21:59:53.949754 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 21:59:53.949754 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 21:59:53.949754 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 21:59:53.949754 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 21:59:53.949754 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 21:59:53.949754 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 21:59:53.949754 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 21:59:53.949754 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 21:59:53.949754 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 21:59:53.949754 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 21:59:53.949754 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 13 21:59:53.949754 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 13 21:59:53.949754 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 13 21:59:53.950753 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 13 21:59:53.950753 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 21:59:53.950753 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 21:59:53.950753 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 21:59:53.950753 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 21:59:53.950753 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 21:59:53.950753 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 21:59:53.950753 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 21:59:53.950753 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application with Lock( Jul 13 21:59:53.950753 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 21:59:53.950753 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._enter() Jul 13 21:59:53.950753 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 21:59:53.950753 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 21:59:53.950753 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 21:59:53.950753 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.creator() Jul 13 21:59:53.950753 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 21:59:53.950753 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application created_value = creator( Jul 13 21:59:53.951761 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 13 21:59:53.951761 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 13 21:59:53.951761 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 13 21:59:53.951761 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 13 21:59:53.951761 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 13 21:59:53.951761 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 13 21:59:53.951761 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 13 21:59:53.951761 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Jul 13 21:59:53.951761 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 419/830] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Jul 13 21:59:53 2022] GET /identity/v3/projects/service => generated 88 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 21:59:53.957250 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5862ab0f-f677-4f2f-860d-f62ccc19c446 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:54.058305 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5862ab0f-f677-4f2f-860d-f62ccc19c446 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:54.091961 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-129127f0-1d21-4f20-89e8-843637639d2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:54.092631 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129127f0-1d21-4f20-89e8-843637639d2f None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:54.092907 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129127f0-1d21-4f20-89e8-843637639d2f None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:54.093177 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-129127f0-1d21-4f20-89e8-843637639d2f None admin] PATH_INFO: `/v3/projects` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:54.093964 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-129127f0-1d21-4f20-89e8-843637639d2f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 21:59:54.094269 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-129127f0-1d21-4f20-89e8-843637639d2f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:54.095735 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-129127f0-1d21-4f20-89e8-843637639d2f None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:54.108675 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 412/831] 10.222.0.52 () {56 vars in 1148 bytes} [Wed Jul 13 21:59:53 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 21:59:54.114481 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2de869-d7a3-4cb8-813c-780a90aba054 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:54.214873 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2de869-d7a3-4cb8-813c-780a90aba054 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:54.235506 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:59:54.254388 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961d0457-b27e-445e-aac4-e11b833efb6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:54.255457 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961d0457-b27e-445e-aac4-e11b833efb6c None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:54.255457 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961d0457-b27e-445e-aac4-e11b833efb6c None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:54.255619 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961d0457-b27e-445e-aac4-e11b833efb6c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:54.256416 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-961d0457-b27e-445e-aac4-e11b833efb6c None admin] RBAC: Adding query filter params (role.id=44efba2579f342df8a0789c59d163ec3, scope.project.id=417e3e2f1333481795a089bef2e24bea, user.id=f84281091687464990ffbca590b9343b) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 21:59:54.256727 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-961d0457-b27e-445e-aac4-e11b833efb6c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:54.258166 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-961d0457-b27e-445e-aac4-e11b833efb6c None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:54.264574 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 420/832] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Jul 13 21:59:54 2022] GET /identity/v3/role_assignments?user.id=f84281091687464990ffbca590b9343b&scope.project.id=417e3e2f1333481795a089bef2e24bea&role.id=44efba2579f342df8a0789c59d163ec3 => generated 608 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 21:59:54.492904 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a25d0b2-6d88-4cc1-9ba7-f4a7aa638ae8 None None] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:54.493334 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a25d0b2-6d88-4cc1-9ba7-f4a7aa638ae8 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:54.493437 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a25d0b2-6d88-4cc1-9ba7-f4a7aa638ae8 None None] PATH_INFO: `/` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:54.495110 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 413/833] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jul 13 21:59:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 21:59:54.500749 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483c92f6-f67a-407e-a04b-88cf8798c7bc None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:54.501611 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483c92f6-f67a-407e-a04b-88cf8798c7bc None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:54.502107 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-483c92f6-f67a-407e-a04b-88cf8798c7bc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:54.519532 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-483c92f6-f67a-407e-a04b-88cf8798c7bc None None] MFA Rules not processed for user `cc99c93da5074cf5a48bc2aa9a92c5e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:59:54.521103 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-483c92f6-f67a-407e-a04b-88cf8798c7bc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:59:54.606465 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 421/834] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jul 13 21:59:54 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 13 21:59:54.626042 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a25d0b2-6d88-4cc1-9ba7-f4a7aa638ae8 None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:54.652555 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:59:54.734073 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a25d0b2-6d88-4cc1-9ba7-f4a7aa638ae8 None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:54.768883 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9d4681-6e88-487e-8531-667797c2b335 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:54.769607 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9d4681-6e88-487e-8531-667797c2b335 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:54.769880 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9d4681-6e88-487e-8531-667797c2b335 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:54.770147 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f9d4681-6e88-487e-8531-667797c2b335 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:54.780741 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f9d4681-6e88-487e-8531-667797c2b335 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 21:59:54.788353 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application [None req-6f9d4681-6e88-487e-8531-667797c2b335 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 13 21:59:54.788353 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 21:59:54.788353 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 21:59:54.788353 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 21:59:54.788353 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 21:59:54.788353 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 21:59:54.788353 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 21:59:54.788353 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 21:59:54.788353 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 21:59:54.788353 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 21:59:54.788353 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 21:59:54.788353 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 21:59:54.788353 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 21:59:54.788353 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 21:59:54.788353 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 21:59:54.788353 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 21:59:54.789282 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 21:59:54.789282 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 21:59:54.789282 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 21:59:54.789282 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 21:59:54.789282 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 21:59:54.789282 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 21:59:54.789282 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 21:59:54.789282 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 21:59:54.789282 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 21:59:54.789282 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 21:59:54.789282 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 21:59:54.789282 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application with Lock( Jul 13 21:59:54.789282 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 21:59:54.789282 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._enter() Jul 13 21:59:54.789282 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 21:59:54.789282 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 21:59:54.790222 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 21:59:54.790222 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.creator() Jul 13 21:59:54.790222 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 21:59:54.790222 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application created_value = creator( Jul 13 21:59:54.790222 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 21:59:54.790222 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 21:59:54.790222 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 21:59:54.790222 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 21:59:54.790222 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 21:59:54.790222 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 21:59:54.790222 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 13 21:59:54.790222 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Jul 13 21:59:54.790222 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 414/835] 10.222.0.52 () {56 vars in 1096 bytes} [Wed Jul 13 21:59:54 2022] GET /identity/v3/roles/admin => generated 83 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 21:59:54.795884 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483c92f6-f67a-407e-a04b-88cf8798c7bc None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:54.903073 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-483c92f6-f67a-407e-a04b-88cf8798c7bc None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:54.917921 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:59:54.940071 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49cc36a5-e996-4ea7-86ed-127f485f147b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:54.941114 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cc36a5-e996-4ea7-86ed-127f485f147b None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:54.941591 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cc36a5-e996-4ea7-86ed-127f485f147b None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:54.942057 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49cc36a5-e996-4ea7-86ed-127f485f147b None admin] PATH_INFO: `/v3/roles` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:54.943072 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49cc36a5-e996-4ea7-86ed-127f485f147b None admin] RBAC: Adding query filter params (name=admin) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 21:59:54.943578 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49cc36a5-e996-4ea7-86ed-127f485f147b None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:54.945173 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49cc36a5-e996-4ea7-86ed-127f485f147b None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:54.955030 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 422/836] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Jul 13 21:59:54 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 21:59:54.962611 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9d4681-6e88-487e-8531-667797c2b335 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:54.964301 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb7ae87-20c7-4855-9b81-2285ffcb9248 None None] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:54.964583 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb7ae87-20c7-4855-9b81-2285ffcb9248 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:54.965797 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddb7ae87-20c7-4855-9b81-2285ffcb9248 None None] PATH_INFO: `/` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:54.967434 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 423/837] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jul 13 21:59:54 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 21:59:54.973199 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba99241-e015-4050-b6f6-8c2ecddc6118 None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:54.973860 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba99241-e015-4050-b6f6-8c2ecddc6118 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:54.974136 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ba99241-e015-4050-b6f6-8c2ecddc6118 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:54.994226 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-9ba99241-e015-4050-b6f6-8c2ecddc6118 None None] MFA Rules not processed for user `cc99c93da5074cf5a48bc2aa9a92c5e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:59:54.996192 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-9ba99241-e015-4050-b6f6-8c2ecddc6118 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:59:55.063880 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:59:55.069697 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 424/838] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jul 13 21:59:54 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 13 21:59:55.091065 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f9d4681-6e88-487e-8531-667797c2b335 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:55.097958 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba99241-e015-4050-b6f6-8c2ecddc6118 None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:55.136724 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6410624b-b0a1-42df-b303-485eb19ee394 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:55.137574 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6410624b-b0a1-42df-b303-485eb19ee394 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:55.137847 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6410624b-b0a1-42df-b303-485eb19ee394 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:55.138116 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6410624b-b0a1-42df-b303-485eb19ee394 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:55.153459 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6410624b-b0a1-42df-b303-485eb19ee394 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:55.155464 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6410624b-b0a1-42df-b303-485eb19ee394 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:55.178822 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application [None req-6410624b-b0a1-42df-b303-485eb19ee394 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 21:59:55.178822 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 21:59:55.178822 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 21:59:55.178822 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 21:59:55.178822 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 21:59:55.178822 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 21:59:55.178822 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 21:59:55.178822 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 21:59:55.178822 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 21:59:55.178822 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 21:59:55.178822 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 21:59:55.178822 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 21:59:55.178822 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 21:59:55.178822 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 21:59:55.178822 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 21:59:55.179789 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 21:59:55.179789 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 21:59:55.179789 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 21:59:55.179789 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 21:59:55.179789 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 21:59:55.179789 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 21:59:55.179789 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 21:59:55.179789 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 21:59:55.179789 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application with Lock( Jul 13 21:59:55.179789 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 21:59:55.179789 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._enter() Jul 13 21:59:55.179789 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 21:59:55.179789 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 21:59:55.179789 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 21:59:55.179789 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.creator() Jul 13 21:59:55.179789 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 21:59:55.179789 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application created_value = creator( Jul 13 21:59:55.180752 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 21:59:55.180752 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 21:59:55.180752 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 21:59:55.180752 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Jul 13 21:59:55.181399 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 415/839] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Jul 13 21:59:54 2022] GET /identity/v3/domains/Default => generated 87 bytes in 221 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 21:59:55.187854 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6410624b-b0a1-42df-b303-485eb19ee394 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:55.213485 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ba99241-e015-4050-b6f6-8c2ecddc6118 None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:55.217463 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:59:55.260516 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb3ba26-7521-4316-83c8-1e3b26388de9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:55.262821 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb3ba26-7521-4316-83c8-1e3b26388de9 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:55.263306 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb3ba26-7521-4316-83c8-1e3b26388de9 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:55.263770 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-deb3ba26-7521-4316-83c8-1e3b26388de9 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:55.277295 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-deb3ba26-7521-4316-83c8-1e3b26388de9 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 21:59:55.289341 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application [None req-deb3ba26-7521-4316-83c8-1e3b26388de9 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 13 21:59:55.289341 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 21:59:55.289341 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 21:59:55.289341 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 21:59:55.289341 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 21:59:55.289341 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 21:59:55.289341 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 21:59:55.289341 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 21:59:55.289341 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 21:59:55.289341 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 21:59:55.289341 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 21:59:55.289341 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 21:59:55.289341 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 21:59:55.289341 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 21:59:55.289341 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 21:59:55.289341 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 21:59:55.291037 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 21:59:55.291037 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 21:59:55.291037 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 21:59:55.291037 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 21:59:55.291037 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 21:59:55.291037 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 21:59:55.291037 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 21:59:55.291037 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 21:59:55.291037 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 21:59:55.291037 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 21:59:55.291037 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 21:59:55.291037 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application with Lock( Jul 13 21:59:55.291037 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 21:59:55.291037 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._enter() Jul 13 21:59:55.291037 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 21:59:55.291037 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 21:59:55.291887 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 21:59:55.291887 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.creator() Jul 13 21:59:55.291887 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 21:59:55.291887 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application created_value = creator( Jul 13 21:59:55.291887 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 21:59:55.291887 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 21:59:55.291887 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 21:59:55.291887 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 21:59:55.291887 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 21:59:55.291887 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 21:59:55.291887 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 13 21:59:55.291887 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Jul 13 21:59:55.297857 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 425/840] 10.222.0.52 () {56 vars in 1102 bytes} [Wed Jul 13 21:59:55 2022] GET /identity/v3/roles/service => generated 85 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 13 21:59:55.299166 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb3ba26-7521-4316-83c8-1e3b26388de9 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:55.302409 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6410624b-b0a1-42df-b303-485eb19ee394 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:55.334715 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:59:55.341417 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ef01ea-2469-4aa5-8031-d49a6e33761d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:55.342127 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ef01ea-2469-4aa5-8031-d49a6e33761d None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:55.342410 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ef01ea-2469-4aa5-8031-d49a6e33761d None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:55.342717 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ef01ea-2469-4aa5-8031-d49a6e33761d None admin] PATH_INFO: `/v3/domains` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:55.343455 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ef01ea-2469-4aa5-8031-d49a6e33761d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 21:59:55.344168 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ef01ea-2469-4aa5-8031-d49a6e33761d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:55.345913 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ef01ea-2469-4aa5-8031-d49a6e33761d None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:55.362880 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 416/841] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Jul 13 21:59:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 21:59:55.362880 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c866de7-b5c2-4587-a6ce-adc001b0ea13 None None] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:55.362880 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c866de7-b5c2-4587-a6ce-adc001b0ea13 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:55.362880 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2c866de7-b5c2-4587-a6ce-adc001b0ea13 None None] PATH_INFO: `/` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:55.362880 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 417/842] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jul 13 21:59:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 21:59:55.367043 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c866de7-b5c2-4587-a6ce-adc001b0ea13 None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:55.408802 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-deb3ba26-7521-4316-83c8-1e3b26388de9 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:55.444408 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb07229b-d0f9-4df0-bee8-d38a64b68b92 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:55.445101 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb07229b-d0f9-4df0-bee8-d38a64b68b92 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:55.445381 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb07229b-d0f9-4df0-bee8-d38a64b68b92 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:55.445657 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb07229b-d0f9-4df0-bee8-d38a64b68b92 None admin] PATH_INFO: `/v3/roles` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:55.446549 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb07229b-d0f9-4df0-bee8-d38a64b68b92 None admin] RBAC: Adding query filter params (name=service) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 21:59:55.446888 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb07229b-d0f9-4df0-bee8-d38a64b68b92 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:55.448333 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb07229b-d0f9-4df0-bee8-d38a64b68b92 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:55.462551 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 426/843] 10.222.0.52 () {56 vars in 1103 bytes} [Wed Jul 13 21:59:55 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 21:59:55.462551 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92247a7e-abbd-4d2c-addd-2c9bae5e8beb None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:55.462551 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92247a7e-abbd-4d2c-addd-2c9bae5e8beb None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:55.462551 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92247a7e-abbd-4d2c-addd-2c9bae5e8beb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:55.479253 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-92247a7e-abbd-4d2c-addd-2c9bae5e8beb None None] MFA Rules not processed for user `cc99c93da5074cf5a48bc2aa9a92c5e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:59:55.480478 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-92247a7e-abbd-4d2c-addd-2c9bae5e8beb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:59:55.484175 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2c866de7-b5c2-4587-a6ce-adc001b0ea13 None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:55.501856 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:59:55.521002 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c0820c-5f52-4543-8702-35dfe3dc04c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:55.522110 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c0820c-5f52-4543-8702-35dfe3dc04c4 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:55.522637 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c0820c-5f52-4543-8702-35dfe3dc04c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:55.523081 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25c0820c-5f52-4543-8702-35dfe3dc04c4 None admin] PATH_INFO: `/v3/users/swift` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:55.531353 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c0820c-5f52-4543-8702-35dfe3dc04c4 None admin] RBAC: Authorizing `identity:get_user(user_id=swift)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:55.533176 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-25c0820c-5f52-4543-8702-35dfe3dc04c4 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:55.549565 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application [None req-25c0820c-5f52-4543-8702-35dfe3dc04c4 None admin] Could not find user: swift.: keystone.exception.UserNotFound: Could not find user: swift. Jul 13 21:59:55.549565 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 21:59:55.549565 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 21:59:55.549565 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 21:59:55.549565 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 21:59:55.549565 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 21:59:55.549565 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 21:59:55.549565 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 21:59:55.549565 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 21:59:55.549565 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 21:59:55.549565 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 21:59:55.549565 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 21:59:55.549565 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 13 21:59:55.549565 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 13 21:59:55.549565 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 13 21:59:55.549565 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 13 21:59:55.550576 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 21:59:55.550576 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 21:59:55.550576 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 13 21:59:55.550576 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 21:59:55.550576 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 13 21:59:55.550576 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 21:59:55.550576 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 21:59:55.550576 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 21:59:55.550576 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 21:59:55.550576 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 21:59:55.550576 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 21:59:55.550576 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application with Lock( Jul 13 21:59:55.550576 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 21:59:55.550576 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._enter() Jul 13 21:59:55.550576 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 21:59:55.550576 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 21:59:55.550576 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 21:59:55.550576 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.creator() Jul 13 21:59:55.551533 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 21:59:55.551533 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application created_value = creator( Jul 13 21:59:55.551533 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 13 21:59:55.551533 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 13 21:59:55.551533 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 13 21:59:55.551533 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 13 21:59:55.551533 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 13 21:59:55.551533 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 13 21:59:55.551533 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: swift. Jul 13 21:59:55.551533 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Jul 13 21:59:55.558615 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 418/844] 10.222.0.52 () {56 vars in 1096 bytes} [Wed Jul 13 21:59:55 2022] GET /identity/v3/users/swift => generated 83 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 21:59:55.558615 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c0820c-5f52-4543-8702-35dfe3dc04c4 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:55.561510 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 427/845] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jul 13 21:59:55 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 13 21:59:55.563486 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92247a7e-abbd-4d2c-addd-2c9bae5e8beb None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:55.624681 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:59:55.676328 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92247a7e-abbd-4d2c-addd-2c9bae5e8beb None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:55.676901 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25c0820c-5f52-4543-8702-35dfe3dc04c4 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:55.714465 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8722941f-b90a-440c-9b41-478b4509f963 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:55.714713 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac82214-4734-4439-b5b6-f768d61ed4e4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:55.715271 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac82214-4734-4439-b5b6-f768d61ed4e4 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:55.715555 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac82214-4734-4439-b5b6-f768d61ed4e4 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:55.715834 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dac82214-4734-4439-b5b6-f768d61ed4e4 None admin] PATH_INFO: `/v3/users` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:55.716225 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8722941f-b90a-440c-9b41-478b4509f963 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:55.716681 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8722941f-b90a-440c-9b41-478b4509f963 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:55.716792 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac82214-4734-4439-b5b6-f768d61ed4e4 None admin] RBAC: Adding query filter params (domain_id=default, name=swift) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 21:59:55.717015 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac82214-4734-4439-b5b6-f768d61ed4e4 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:55.717431 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8722941f-b90a-440c-9b41-478b4509f963 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:55.718459 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dac82214-4734-4439-b5b6-f768d61ed4e4 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:55.729472 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8722941f-b90a-440c-9b41-478b4509f963 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:55.731232 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8722941f-b90a-440c-9b41-478b4509f963 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:55.736266 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 428/846] 10.222.0.52 () {56 vars in 1135 bytes} [Wed Jul 13 21:59:55 2022] GET /identity/v3/users?domain_id=default&name=swift => generated 370 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 21:59:55.737074 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac82214-4734-4439-b5b6-f768d61ed4e4 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:55.752339 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application [None req-8722941f-b90a-440c-9b41-478b4509f963 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 21:59:55.752339 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 21:59:55.752339 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 21:59:55.752339 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 21:59:55.752339 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 21:59:55.752339 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 21:59:55.752339 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 21:59:55.752339 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 21:59:55.752339 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 21:59:55.752339 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 21:59:55.752339 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 21:59:55.752339 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 21:59:55.752339 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 21:59:55.752339 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 21:59:55.752339 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 21:59:55.753248 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 21:59:55.753248 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 21:59:55.753248 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 21:59:55.753248 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 21:59:55.753248 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 21:59:55.753248 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 21:59:55.753248 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 21:59:55.753248 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 21:59:55.753248 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application with Lock( Jul 13 21:59:55.753248 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 21:59:55.753248 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._enter() Jul 13 21:59:55.753248 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 21:59:55.753248 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 21:59:55.753248 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 21:59:55.753248 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.creator() Jul 13 21:59:55.753248 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 21:59:55.753248 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application created_value = creator( Jul 13 21:59:55.754134 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 21:59:55.754134 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 21:59:55.754134 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 21:59:55.754134 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Jul 13 21:59:55.754134 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 419/847] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Jul 13 21:59:55 2022] GET /identity/v3/domains/Default => generated 87 bytes in 199 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 21:59:55.754427 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8722941f-b90a-440c-9b41-478b4509f963 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:55.759481 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:59:55.846159 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dac82214-4734-4439-b5b6-f768d61ed4e4 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:55.872338 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8722941f-b90a-440c-9b41-478b4509f963 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:55.876160 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:59:55.886900 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0fc23f6-d820-4046-8cf0-c2dee75dd8e2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:55.887706 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fc23f6-d820-4046-8cf0-c2dee75dd8e2 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:55.887983 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fc23f6-d820-4046-8cf0-c2dee75dd8e2 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:55.888248 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0fc23f6-d820-4046-8cf0-c2dee75dd8e2 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:55.889003 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0fc23f6-d820-4046-8cf0-c2dee75dd8e2 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:55.890277 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0fc23f6-d820-4046-8cf0-c2dee75dd8e2 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:55.902524 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application [None req-a0fc23f6-d820-4046-8cf0-c2dee75dd8e2 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 13 21:59:55.902524 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 21:59:55.902524 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 13 21:59:55.902524 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 13 21:59:55.902524 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 13 21:59:55.902524 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 13 21:59:55.902524 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 13 21:59:55.902524 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 13 21:59:55.902524 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 13 21:59:55.902524 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Jul 13 21:59:55.902524 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 13 21:59:55.902524 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Jul 13 21:59:55.902524 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 21:59:55.902524 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 21:59:55.902524 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 21:59:55.902524 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 21:59:55.902524 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 21:59:55.903548 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 21:59:55.903548 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 21:59:55.903548 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 21:59:55.903548 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 21:59:55.903548 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 21:59:55.903548 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 21:59:55.903548 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 13 21:59:55.903548 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 13 21:59:55.903548 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 13 21:59:55.903548 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 13 21:59:55.903548 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 21:59:55.903548 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 21:59:55.903548 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 21:59:55.903548 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 21:59:55.903548 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 21:59:55.903548 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 21:59:55.904387 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 21:59:55.904387 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application with Lock( Jul 13 21:59:55.904387 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 21:59:55.904387 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._enter() Jul 13 21:59:55.904387 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 21:59:55.904387 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 21:59:55.904387 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 21:59:55.904387 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.creator() Jul 13 21:59:55.904387 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 21:59:55.904387 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application created_value = creator( Jul 13 21:59:55.904387 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 13 21:59:55.904387 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 13 21:59:55.904387 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 13 21:59:55.904387 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Jul 13 21:59:55.904387 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 429/848] 10.222.0.52 () {56 vars in 1114 bytes} [Wed Jul 13 21:59:55 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 21:59:55.905201 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0fc23f6-d820-4046-8cf0-c2dee75dd8e2 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:55.918744 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92938ea3-51d0-4218-aa28-af19dc372fd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:55.919411 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92938ea3-51d0-4218-aa28-af19dc372fd3 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:55.919718 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92938ea3-51d0-4218-aa28-af19dc372fd3 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:55.919964 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92938ea3-51d0-4218-aa28-af19dc372fd3 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:55.988172 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:59:55.995755 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92938ea3-51d0-4218-aa28-af19dc372fd3 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:55.997254 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92938ea3-51d0-4218-aa28-af19dc372fd3 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:56.011930 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0fc23f6-d820-4046-8cf0-c2dee75dd8e2 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:56.046947 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b83600-8575-4926-b263-d633ec2bdbce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:56.047621 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b83600-8575-4926-b263-d633ec2bdbce None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:56.047909 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b83600-8575-4926-b263-d633ec2bdbce None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:56.048179 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59b83600-8575-4926-b263-d633ec2bdbce None admin] PATH_INFO: `/v3/domains` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:56.048906 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b83600-8575-4926-b263-d633ec2bdbce None admin] RBAC: Adding query filter params (name=Default) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 21:59:56.049212 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b83600-8575-4926-b263-d633ec2bdbce None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:56.050614 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59b83600-8575-4926-b263-d633ec2bdbce None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:56.068287 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 430/849] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Jul 13 21:59:55 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 21:59:56.068287 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b83600-8575-4926-b263-d633ec2bdbce None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:56.106545 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 420/850] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Jul 13 21:59:55 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 13 21:59:56.111635 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92938ea3-51d0-4218-aa28-af19dc372fd3 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:56.113271 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:59:56.175488 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59b83600-8575-4926-b263-d633ec2bdbce None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:56.212145 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4566b57f-be8f-4103-93ba-5b931bdead24 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:56.213258 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4566b57f-be8f-4103-93ba-5b931bdead24 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:56.213741 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4566b57f-be8f-4103-93ba-5b931bdead24 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:56.214222 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4566b57f-be8f-4103-93ba-5b931bdead24 None admin] PATH_INFO: `/v3/services` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:56.214439 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92938ea3-51d0-4218-aa28-af19dc372fd3 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:56.215066 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4566b57f-be8f-4103-93ba-5b931bdead24 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 21:59:56.215379 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4566b57f-be8f-4103-93ba-5b931bdead24 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:56.216775 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4566b57f-be8f-4103-93ba-5b931bdead24 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:56.221405 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 431/851] 10.222.0.52 () {56 vars in 1114 bytes} [Wed Jul 13 21:59:56 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 21:59:56.222217 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4566b57f-be8f-4103-93ba-5b931bdead24 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:56.225065 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:59:56.261771 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b083236-853d-4fa6-a7e7-a84df75bd64f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:56.262625 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b083236-853d-4fa6-a7e7-a84df75bd64f None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:56.262909 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b083236-853d-4fa6-a7e7-a84df75bd64f None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:56.263182 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b083236-853d-4fa6-a7e7-a84df75bd64f None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:56.270578 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b083236-853d-4fa6-a7e7-a84df75bd64f None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:56.272023 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b083236-853d-4fa6-a7e7-a84df75bd64f None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:56.287880 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application [None req-6b083236-853d-4fa6-a7e7-a84df75bd64f None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 13 21:59:56.287880 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 21:59:56.287880 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 21:59:56.287880 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 21:59:56.287880 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 21:59:56.287880 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 21:59:56.287880 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 21:59:56.287880 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 21:59:56.287880 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 21:59:56.287880 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 21:59:56.287880 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 21:59:56.287880 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 21:59:56.287880 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 13 21:59:56.287880 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 13 21:59:56.287880 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 13 21:59:56.287880 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 13 21:59:56.289199 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 21:59:56.289199 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 21:59:56.289199 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 13 21:59:56.289199 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 21:59:56.289199 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 13 21:59:56.289199 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 21:59:56.289199 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 21:59:56.289199 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 21:59:56.289199 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 21:59:56.289199 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 21:59:56.289199 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 21:59:56.289199 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application with Lock( Jul 13 21:59:56.289199 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 21:59:56.289199 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._enter() Jul 13 21:59:56.289199 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 21:59:56.289199 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 21:59:56.289199 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 21:59:56.289199 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.creator() Jul 13 21:59:56.290442 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 21:59:56.290442 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application created_value = creator( Jul 13 21:59:56.290442 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 13 21:59:56.290442 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 13 21:59:56.290442 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 13 21:59:56.290442 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 13 21:59:56.290442 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 13 21:59:56.290442 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 13 21:59:56.290442 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 13 21:59:56.290442 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Jul 13 21:59:56.290442 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 421/852] 10.222.0.52 () {56 vars in 1117 bytes} [Wed Jul 13 21:59:56 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 21:59:56.291094 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b083236-853d-4fa6-a7e7-a84df75bd64f None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:56.335771 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4566b57f-be8f-4103-93ba-5b931bdead24 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:56.356360 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:59:56.371797 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0294cb-378f-4fa2-b8b7-33b5a0faffd0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:56.372900 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0294cb-378f-4fa2-b8b7-33b5a0faffd0 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:56.373175 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0294cb-378f-4fa2-b8b7-33b5a0faffd0 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:56.373440 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3b0294cb-378f-4fa2-b8b7-33b5a0faffd0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:56.384571 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0294cb-378f-4fa2-b8b7-33b5a0faffd0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:56.386426 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3b0294cb-378f-4fa2-b8b7-33b5a0faffd0 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:56.401357 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b083236-853d-4fa6-a7e7-a84df75bd64f None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:56.404065 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application [None req-3b0294cb-378f-4fa2-b8b7-33b5a0faffd0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 21:59:56.404065 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 21:59:56.404065 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 21:59:56.404065 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 21:59:56.404065 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 21:59:56.404065 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 21:59:56.404065 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 21:59:56.404065 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 21:59:56.404065 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 21:59:56.404065 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 21:59:56.404065 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 21:59:56.404065 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 21:59:56.404065 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 21:59:56.404065 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 21:59:56.404065 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 21:59:56.404981 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 21:59:56.404981 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 21:59:56.404981 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 21:59:56.404981 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 21:59:56.404981 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 21:59:56.404981 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 21:59:56.404981 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 21:59:56.404981 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 21:59:56.404981 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application with Lock( Jul 13 21:59:56.404981 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 21:59:56.404981 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._enter() Jul 13 21:59:56.404981 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 21:59:56.404981 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 21:59:56.404981 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 21:59:56.404981 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.creator() Jul 13 21:59:56.404981 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 21:59:56.404981 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application created_value = creator( Jul 13 21:59:56.405885 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 21:59:56.405885 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 21:59:56.405885 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 21:59:56.405885 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Jul 13 21:59:56.405885 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 432/853] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Jul 13 21:59:56 2022] GET /identity/v3/domains/Default => generated 87 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 21:59:56.406175 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0294cb-378f-4fa2-b8b7-33b5a0faffd0 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:56.437032 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af0d94e8-e048-4671-952a-995f34554277 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:56.437615 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0d94e8-e048-4671-952a-995f34554277 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:56.437898 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0d94e8-e048-4671-952a-995f34554277 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:56.438172 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af0d94e8-e048-4671-952a-995f34554277 None admin] PATH_INFO: `/v3/services` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:56.438916 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0d94e8-e048-4671-952a-995f34554277 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 21:59:56.439220 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0d94e8-e048-4671-952a-995f34554277 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:56.440595 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af0d94e8-e048-4671-952a-995f34554277 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:56.445029 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 422/854] 10.222.0.52 () {56 vars in 1114 bytes} [Wed Jul 13 21:59:56 2022] GET /identity/v3/services?type=volumev3 => generated 125 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 21:59:56.446537 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec46746-5b80-404d-990a-beff351678f3 None None] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:56.446892 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec46746-5b80-404d-990a-beff351678f3 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:56.447170 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dec46746-5b80-404d-990a-beff351678f3 None None] PATH_INFO: `/` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:56.448428 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 423/855] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jul 13 21:59:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 21:59:56.449489 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec46746-5b80-404d-990a-beff351678f3 None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:56.487303 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:59:56.549243 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3b0294cb-378f-4fa2-b8b7-33b5a0faffd0 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:56.575114 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dec46746-5b80-404d-990a-beff351678f3 None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:56.590441 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a98c49b-8a46-4c27-82a2-82f96c248949 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:56.591169 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a98c49b-8a46-4c27-82a2-82f96c248949 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:56.591430 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a98c49b-8a46-4c27-82a2-82f96c248949 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:56.591705 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a98c49b-8a46-4c27-82a2-82f96c248949 None admin] PATH_INFO: `/v3/users` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:56.592560 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a98c49b-8a46-4c27-82a2-82f96c248949 None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 21:59:56.592863 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a98c49b-8a46-4c27-82a2-82f96c248949 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:56.594332 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a98c49b-8a46-4c27-82a2-82f96c248949 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:56.606411 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:59:56.615276 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 433/856] 10.222.0.52 () {56 vars in 1149 bytes} [Wed Jul 13 21:59:56 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 209 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 21:59:56.616105 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f60fffe-155b-4cf7-b136-ea029813551f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:56.616568 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f51f87-d9c1-4dd3-8fbe-c1ddcb555387 None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:56.616834 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f60fffe-155b-4cf7-b136-ea029813551f None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:56.616937 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f51f87-d9c1-4dd3-8fbe-c1ddcb555387 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:56.617119 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f60fffe-155b-4cf7-b136-ea029813551f None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:56.617218 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98f51f87-d9c1-4dd3-8fbe-c1ddcb555387 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:56.617393 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f60fffe-155b-4cf7-b136-ea029813551f None admin] PATH_INFO: `/v3/domains` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:56.618183 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f60fffe-155b-4cf7-b136-ea029813551f None admin] RBAC: Adding query filter params (name=Default) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 21:59:56.618502 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f60fffe-155b-4cf7-b136-ea029813551f None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:56.620410 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f60fffe-155b-4cf7-b136-ea029813551f None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:56.632601 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 424/857] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Jul 13 21:59:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 21:59:56.633244 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f60fffe-155b-4cf7-b136-ea029813551f None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:56.635723 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-98f51f87-d9c1-4dd3-8fbe-c1ddcb555387 None None] MFA Rules not processed for user `cc99c93da5074cf5a48bc2aa9a92c5e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:59:56.637670 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-98f51f87-d9c1-4dd3-8fbe-c1ddcb555387 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:59:56.703601 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 434/858] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jul 13 21:59:56 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 13 21:59:56.704356 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f51f87-d9c1-4dd3-8fbe-c1ddcb555387 None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:56.744352 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f60fffe-155b-4cf7-b136-ea029813551f None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:56.755252 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:59:56.783673 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1319ff8-3152-4859-b164-b4812178fcc6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:56.784330 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1319ff8-3152-4859-b164-b4812178fcc6 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:56.784605 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1319ff8-3152-4859-b164-b4812178fcc6 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:56.784873 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1319ff8-3152-4859-b164-b4812178fcc6 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:56.821687 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98f51f87-d9c1-4dd3-8fbe-c1ddcb555387 None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:56.860804 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26489169-09a5-4b1f-adca-55d6fb5f4b6f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:56.862010 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26489169-09a5-4b1f-adca-55d6fb5f4b6f None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:56.862638 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26489169-09a5-4b1f-adca-55d6fb5f4b6f None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:56.863156 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26489169-09a5-4b1f-adca-55d6fb5f4b6f None admin] PATH_INFO: `/v3/projects/service` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:56.871543 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26489169-09a5-4b1f-adca-55d6fb5f4b6f None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:56.873390 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26489169-09a5-4b1f-adca-55d6fb5f4b6f None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:56.888866 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application [None req-26489169-09a5-4b1f-adca-55d6fb5f4b6f None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 13 21:59:56.888866 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 21:59:56.888866 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 21:59:56.888866 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 21:59:56.888866 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 21:59:56.888866 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 21:59:56.888866 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 21:59:56.888866 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 21:59:56.888866 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 21:59:56.888866 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 21:59:56.888866 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 21:59:56.888866 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 21:59:56.888866 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 13 21:59:56.888866 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 13 21:59:56.888866 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 13 21:59:56.889899 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 13 21:59:56.889899 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 21:59:56.889899 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 21:59:56.889899 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 21:59:56.889899 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 21:59:56.889899 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 21:59:56.889899 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 21:59:56.889899 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 21:59:56.889899 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application with Lock( Jul 13 21:59:56.889899 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 21:59:56.889899 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._enter() Jul 13 21:59:56.889899 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 21:59:56.889899 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 21:59:56.889899 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 21:59:56.889899 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.creator() Jul 13 21:59:56.889899 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 21:59:56.889899 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application created_value = creator( Jul 13 21:59:56.891040 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 13 21:59:56.891040 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 13 21:59:56.891040 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 13 21:59:56.891040 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 13 21:59:56.891040 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 13 21:59:56.891040 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 13 21:59:56.891040 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 13 21:59:56.891040 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Jul 13 21:59:56.892943 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 435/859] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Jul 13 21:59:56 2022] GET /identity/v3/projects/service => generated 88 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 21:59:56.893993 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26489169-09a5-4b1f-adca-55d6fb5f4b6f None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:56.964334 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:59:56.977694 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1319ff8-3152-4859-b164-b4812178fcc6 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:56.979441 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1319ff8-3152-4859-b164-b4812178fcc6 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:57.013609 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26489169-09a5-4b1f-adca-55d6fb5f4b6f None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:57.015559 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:59:57.050953 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d4c99e-4218-43d7-b10c-19cfbf6c4ee2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:57.052011 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d4c99e-4218-43d7-b10c-19cfbf6c4ee2 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:57.052469 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d4c99e-4218-43d7-b10c-19cfbf6c4ee2 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:57.052892 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72d4c99e-4218-43d7-b10c-19cfbf6c4ee2 None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:57.053688 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d4c99e-4218-43d7-b10c-19cfbf6c4ee2 None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:57.055265 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72d4c99e-4218-43d7-b10c-19cfbf6c4ee2 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:57.067684 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application [None req-72d4c99e-4218-43d7-b10c-19cfbf6c4ee2 None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 13 21:59:57.067684 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 21:59:57.067684 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 13 21:59:57.067684 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 13 21:59:57.067684 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 13 21:59:57.067684 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 13 21:59:57.067684 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 13 21:59:57.067684 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 13 21:59:57.067684 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 13 21:59:57.067684 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Jul 13 21:59:57.067684 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 13 21:59:57.067684 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Jul 13 21:59:57.067684 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 21:59:57.067684 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 21:59:57.067684 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 21:59:57.067684 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 21:59:57.068878 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 21:59:57.068878 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 21:59:57.068878 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 21:59:57.068878 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 21:59:57.068878 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 21:59:57.068878 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 21:59:57.068878 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 21:59:57.068878 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 13 21:59:57.068878 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 13 21:59:57.068878 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 13 21:59:57.068878 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 13 21:59:57.068878 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 21:59:57.068878 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 21:59:57.068878 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 21:59:57.068878 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 21:59:57.068878 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 21:59:57.069838 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 21:59:57.069838 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 21:59:57.069838 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application with Lock( Jul 13 21:59:57.069838 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 21:59:57.069838 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._enter() Jul 13 21:59:57.069838 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 21:59:57.069838 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 21:59:57.069838 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 21:59:57.069838 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.creator() Jul 13 21:59:57.069838 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 21:59:57.069838 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application created_value = creator( Jul 13 21:59:57.069838 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 13 21:59:57.069838 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 13 21:59:57.069838 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 13 21:59:57.069838 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Jul 13 21:59:57.072263 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 436/860] 10.222.0.52 () {56 vars in 1132 bytes} [Wed Jul 13 21:59:56 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 21:59:57.073206 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d4c99e-4218-43d7-b10c-19cfbf6c4ee2 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:57.088660 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 425/861] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Jul 13 21:59:56 2022] GET /identity/v3/auth/tokens => generated 1151 bytes in 456 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 13 21:59:57.089787 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1319ff8-3152-4859-b164-b4812178fcc6 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:57.181151 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:59:57.184454 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72d4c99e-4218-43d7-b10c-19cfbf6c4ee2 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:57.195366 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1319ff8-3152-4859-b164-b4812178fcc6 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:57.220015 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4948d6d8-58bb-489f-bed6-71d8b99d1ae8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:57.220710 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4948d6d8-58bb-489f-bed6-71d8b99d1ae8 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:57.220913 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4948d6d8-58bb-489f-bed6-71d8b99d1ae8 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:57.221187 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4948d6d8-58bb-489f-bed6-71d8b99d1ae8 None admin] PATH_INFO: `/v3/projects` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:57.221977 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4948d6d8-58bb-489f-bed6-71d8b99d1ae8 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 21:59:57.222285 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4948d6d8-58bb-489f-bed6-71d8b99d1ae8 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:57.223751 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4948d6d8-58bb-489f-bed6-71d8b99d1ae8 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:57.230117 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95eddc48-f244-48fc-a1fb-9ec7d3f59702 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:57.230872 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95eddc48-f244-48fc-a1fb-9ec7d3f59702 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:57.231154 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95eddc48-f244-48fc-a1fb-9ec7d3f59702 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:57.231422 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95eddc48-f244-48fc-a1fb-9ec7d3f59702 None admin] PATH_INFO: `/v3/services` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:57.232122 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95eddc48-f244-48fc-a1fb-9ec7d3f59702 None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 21:59:57.232440 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95eddc48-f244-48fc-a1fb-9ec7d3f59702 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:57.233844 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95eddc48-f244-48fc-a1fb-9ec7d3f59702 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:57.240679 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 437/862] 10.222.0.52 () {56 vars in 1148 bytes} [Wed Jul 13 21:59:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 21:59:57.240679 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4948d6d8-58bb-489f-bed6-71d8b99d1ae8 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:57.243697 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 426/863] 10.222.0.52 () {56 vars in 1126 bytes} [Wed Jul 13 21:59:57 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 21:59:57.244192 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95eddc48-f244-48fc-a1fb-9ec7d3f59702 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:57.269827 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:59:57.343979 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95eddc48-f244-48fc-a1fb-9ec7d3f59702 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:57.344175 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4948d6d8-58bb-489f-bed6-71d8b99d1ae8 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:57.378834 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3589044b-8491-47e3-af3a-79dc3c6c591b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:57.379490 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3589044b-8491-47e3-af3a-79dc3c6c591b None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:57.379701 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c657331-146e-4e2c-a713-e248cdbb9b80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:57.379825 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3589044b-8491-47e3-af3a-79dc3c6c591b None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:57.380019 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3589044b-8491-47e3-af3a-79dc3c6c591b None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:57.380495 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c657331-146e-4e2c-a713-e248cdbb9b80 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:57.380776 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c657331-146e-4e2c-a713-e248cdbb9b80 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:57.380867 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3589044b-8491-47e3-af3a-79dc3c6c591b None admin] RBAC: Adding query filter params (role.id=44efba2579f342df8a0789c59d163ec3, scope.project.id=417e3e2f1333481795a089bef2e24bea, user.id=1eb1525233ba48c99b15d4cb75db4822) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 21:59:57.381050 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c657331-146e-4e2c-a713-e248cdbb9b80 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:57.381142 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3589044b-8491-47e3-af3a-79dc3c6c591b None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:57.382578 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3589044b-8491-47e3-af3a-79dc3c6c591b None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:57.398534 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 427/864] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Jul 13 21:59:57 2022] GET /identity/v3/role_assignments?user.id=1eb1525233ba48c99b15d4cb75db4822&scope.project.id=417e3e2f1333481795a089bef2e24bea&role.id=44efba2579f342df8a0789c59d163ec3 => generated 608 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 21:59:57.398534 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3589044b-8491-47e3-af3a-79dc3c6c591b None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:57.402515 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c657331-146e-4e2c-a713-e248cdbb9b80 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:57.404041 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c657331-146e-4e2c-a713-e248cdbb9b80 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:57.417135 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:59:57.424976 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application [None req-3c657331-146e-4e2c-a713-e248cdbb9b80 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 21:59:57.424976 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 21:59:57.424976 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 21:59:57.424976 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 21:59:57.424976 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 21:59:57.424976 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 21:59:57.424976 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 21:59:57.424976 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 21:59:57.424976 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 21:59:57.424976 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 21:59:57.424976 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 21:59:57.424976 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 21:59:57.424976 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 21:59:57.424976 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 21:59:57.424976 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 21:59:57.426007 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 21:59:57.426007 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 21:59:57.426007 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 21:59:57.426007 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 21:59:57.426007 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 21:59:57.426007 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 21:59:57.426007 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 21:59:57.426007 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 21:59:57.426007 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application with Lock( Jul 13 21:59:57.426007 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 21:59:57.426007 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._enter() Jul 13 21:59:57.426007 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 21:59:57.426007 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 21:59:57.426007 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 21:59:57.426007 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.creator() Jul 13 21:59:57.426007 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 21:59:57.426007 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application created_value = creator( Jul 13 21:59:57.426950 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 21:59:57.426950 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 21:59:57.426950 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 21:59:57.426950 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Jul 13 21:59:57.427594 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 438/865] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Jul 13 21:59:57 2022] GET /identity/v3/domains/Default => generated 87 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 21:59:57.432928 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c657331-146e-4e2c-a713-e248cdbb9b80 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:57.529412 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:59:57.543906 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3589044b-8491-47e3-af3a-79dc3c6c591b None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:57.571307 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c657331-146e-4e2c-a713-e248cdbb9b80 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:57.597362 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d68a29-8db7-44a8-9c13-6e32f3445713 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:57.598116 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d68a29-8db7-44a8-9c13-6e32f3445713 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:57.598315 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d68a29-8db7-44a8-9c13-6e32f3445713 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:57.599163 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07d68a29-8db7-44a8-9c13-6e32f3445713 None admin] PATH_INFO: `/v3/services` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:57.599436 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d68a29-8db7-44a8-9c13-6e32f3445713 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 21:59:57.600125 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d68a29-8db7-44a8-9c13-6e32f3445713 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:57.601147 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07d68a29-8db7-44a8-9c13-6e32f3445713 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:57.607202 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 428/866] 10.222.0.52 () {56 vars in 1126 bytes} [Wed Jul 13 21:59:57 2022] GET /identity/v3/services?type=compute_legacy => generated 131 bytes in 218 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 21:59:57.613678 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790653e6-4d3d-4de2-8ab3-612c2a17cccc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:57.614378 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790653e6-4d3d-4de2-8ab3-612c2a17cccc None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:57.614690 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790653e6-4d3d-4de2-8ab3-612c2a17cccc None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:57.614964 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-790653e6-4d3d-4de2-8ab3-612c2a17cccc None admin] PATH_INFO: `/v3/domains` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:57.615709 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790653e6-4d3d-4de2-8ab3-612c2a17cccc None admin] RBAC: Adding query filter params (name=Default) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 21:59:57.616019 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790653e6-4d3d-4de2-8ab3-612c2a17cccc None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:57.617431 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-790653e6-4d3d-4de2-8ab3-612c2a17cccc None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:57.632559 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 439/867] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Jul 13 21:59:57 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 21:59:57.638754 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d68a29-8db7-44a8-9c13-6e32f3445713 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:57.734099 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:59:57.755367 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07d68a29-8db7-44a8-9c13-6e32f3445713 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:57.792909 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3faa27e-b9d1-4d0b-b8f1-6e19d81655c6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:57.793686 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3faa27e-b9d1-4d0b-b8f1-6e19d81655c6 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:57.793987 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3faa27e-b9d1-4d0b-b8f1-6e19d81655c6 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:57.794284 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3faa27e-b9d1-4d0b-b8f1-6e19d81655c6 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:57.801691 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3faa27e-b9d1-4d0b-b8f1-6e19d81655c6 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:57.803185 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3faa27e-b9d1-4d0b-b8f1-6e19d81655c6 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:57.819329 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application [None req-c3faa27e-b9d1-4d0b-b8f1-6e19d81655c6 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 13 21:59:57.819329 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 21:59:57.819329 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 21:59:57.819329 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 21:59:57.819329 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 21:59:57.819329 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 21:59:57.819329 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 21:59:57.819329 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 21:59:57.819329 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 21:59:57.819329 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 21:59:57.819329 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 21:59:57.819329 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 21:59:57.819329 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 13 21:59:57.819329 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 13 21:59:57.819329 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 13 21:59:57.820271 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 13 21:59:57.820271 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 21:59:57.820271 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 21:59:57.820271 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 21:59:57.820271 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 21:59:57.820271 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 21:59:57.820271 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 21:59:57.820271 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 21:59:57.820271 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application with Lock( Jul 13 21:59:57.820271 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 21:59:57.820271 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._enter() Jul 13 21:59:57.820271 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 21:59:57.820271 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 21:59:57.820271 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 21:59:57.820271 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.creator() Jul 13 21:59:57.820271 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 21:59:57.820271 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application created_value = creator( Jul 13 21:59:57.821275 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 13 21:59:57.821275 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 13 21:59:57.821275 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 13 21:59:57.821275 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 13 21:59:57.821275 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 13 21:59:57.821275 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 13 21:59:57.821275 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 13 21:59:57.821275 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Jul 13 21:59:57.821275 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 429/868] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Jul 13 21:59:57 2022] GET /identity/v3/projects/service => generated 88 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 21:59:57.828414 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790653e6-4d3d-4de2-8ab3-612c2a17cccc None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:57.938852 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-790653e6-4d3d-4de2-8ab3-612c2a17cccc None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:57.976585 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8a7ad29-8f4c-40ec-8a07-b71a784dccdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:57.978045 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a7ad29-8f4c-40ec-8a07-b71a784dccdd None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:57.978526 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a7ad29-8f4c-40ec-8a07-b71a784dccdd None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:57.979051 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8a7ad29-8f4c-40ec-8a07-b71a784dccdd None admin] PATH_INFO: `/v3/projects` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:57.980216 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a7ad29-8f4c-40ec-8a07-b71a784dccdd None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 21:59:57.980704 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a7ad29-8f4c-40ec-8a07-b71a784dccdd None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:57.982325 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8a7ad29-8f4c-40ec-8a07-b71a784dccdd None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:57.990236 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:59:58.002427 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 440/869] 10.222.0.52 () {56 vars in 1148 bytes} [Wed Jul 13 21:59:57 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 21:59:58.004164 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3faa27e-b9d1-4d0b-b8f1-6e19d81655c6 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:58.108868 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3faa27e-b9d1-4d0b-b8f1-6e19d81655c6 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:58.145645 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c25ce10c-c3b0-4ade-bb5d-90f2e9618997 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:58.146648 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25ce10c-c3b0-4ade-bb5d-90f2e9618997 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:58.147122 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25ce10c-c3b0-4ade-bb5d-90f2e9618997 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:58.147590 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c25ce10c-c3b0-4ade-bb5d-90f2e9618997 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:58.148611 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c25ce10c-c3b0-4ade-bb5d-90f2e9618997 None admin] RBAC: Adding query filter params (role.id=70c34ea493484322911dab62b4a56e75, scope.project.id=417e3e2f1333481795a089bef2e24bea, user.id=b866bacdafae4db6a32714c07c56515f) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 21:59:58.149247 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c25ce10c-c3b0-4ade-bb5d-90f2e9618997 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:58.151566 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c25ce10c-c3b0-4ade-bb5d-90f2e9618997 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:58.162103 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 430/870] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Jul 13 21:59:58 2022] GET /identity/v3/role_assignments?user.id=b866bacdafae4db6a32714c07c56515f&scope.project.id=417e3e2f1333481795a089bef2e24bea&role.id=70c34ea493484322911dab62b4a56e75 => generated 608 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 21:59:58.471815 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b785eb-94c4-4e43-9dba-7c99056ab2f4 None None] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:58.472208 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b785eb-94c4-4e43-9dba-7c99056ab2f4 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:58.472381 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b785eb-94c4-4e43-9dba-7c99056ab2f4 None None] PATH_INFO: `/` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:58.473885 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 441/871] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jul 13 21:59:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 21:59:58.479406 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db49311e-98f1-4da1-a4e9-d86434287b7c None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:58.479936 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db49311e-98f1-4da1-a4e9-d86434287b7c None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:58.480413 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db49311e-98f1-4da1-a4e9-d86434287b7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:58.486903 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:59:58.498027 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-db49311e-98f1-4da1-a4e9-d86434287b7c None None] MFA Rules not processed for user `cc99c93da5074cf5a48bc2aa9a92c5e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:59:58.500039 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-db49311e-98f1-4da1-a4e9-d86434287b7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:59:58.573309 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 431/872] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jul 13 21:59:58 2022] POST /identity/v3/auth/tokens => generated 1151 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 13 21:59:58.596219 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b785eb-94c4-4e43-9dba-7c99056ab2f4 None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:58.706073 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b785eb-94c4-4e43-9dba-7c99056ab2f4 None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:58.740794 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c6fa921-8d87-4208-a541-329144f7af2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:58.741396 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6fa921-8d87-4208-a541-329144f7af2f None admin] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:58.741679 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6fa921-8d87-4208-a541-329144f7af2f None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:58.741959 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c6fa921-8d87-4208-a541-329144f7af2f None admin] PATH_INFO: `/v3/services` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:58.742756 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6fa921-8d87-4208-a541-329144f7af2f None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:58.744144 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c6fa921-8d87-4208-a541-329144f7af2f None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:58.765300 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 442/873] 10.222.0.52 () {60 vars in 1141 bytes} [Wed Jul 13 21:59:58 2022] POST /identity/v3/services => generated 251 bytes in 169 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 21:59:59.647439 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e12779-7845-422d-a044-afbf8735549c None None] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:59.648002 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e12779-7845-422d-a044-afbf8735549c None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:59.648230 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81e12779-7845-422d-a044-afbf8735549c None None] PATH_INFO: `/` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:59.650105 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 432/874] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jul 13 21:59:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 21:59:59.655831 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4340d9cf-177b-49ab-ae5a-8ba8b93669f2 None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:59.656403 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4340d9cf-177b-49ab-ae5a-8ba8b93669f2 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:59.656934 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4340d9cf-177b-49ab-ae5a-8ba8b93669f2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:59.681955 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-4340d9cf-177b-49ab-ae5a-8ba8b93669f2 None None] MFA Rules not processed for user `cc99c93da5074cf5a48bc2aa9a92c5e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:59:59.683979 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-4340d9cf-177b-49ab-ae5a-8ba8b93669f2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:59:59.737244 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff4b1b8-37ce-4d0f-b9e3-d26b513a2cf4 None None] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:59.737561 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff4b1b8-37ce-4d0f-b9e3-d26b513a2cf4 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:59.737795 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff4b1b8-37ce-4d0f-b9e3-d26b513a2cf4 None None] PATH_INFO: `/` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:59.739425 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 433/875] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jul 13 21:59:59 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 21:59:59.743461 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:59:59.744516 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39ac111-aea5-411b-a11b-9f07aa1000fb None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:59.744797 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39ac111-aea5-411b-a11b-9f07aa1000fb None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:59.745070 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f39ac111-aea5-411b-a11b-9f07aa1000fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:59.761772 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-f39ac111-aea5-411b-a11b-9f07aa1000fb None None] MFA Rules not processed for user `cc99c93da5074cf5a48bc2aa9a92c5e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 21:59:59.763001 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-f39ac111-aea5-411b-a11b-9f07aa1000fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 21:59:59.770749 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 443/876] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jul 13 21:59:59 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 13 21:59:59.787407 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4340d9cf-177b-49ab-ae5a-8ba8b93669f2 None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:59.831973 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 434/877] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jul 13 21:59:59 2022] POST /identity/v3/auth/tokens => generated 1252 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 13 21:59:59.855281 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39ac111-aea5-411b-a11b-9f07aa1000fb None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:59.891237 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4340d9cf-177b-49ab-ae5a-8ba8b93669f2 None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:59.926954 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 21:59:59.928171 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5806f775-0b84-4e74-89db-0972c4d059c4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:59.929006 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5806f775-0b84-4e74-89db-0972c4d059c4 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 21:59:59.929288 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5806f775-0b84-4e74-89db-0972c4d059c4 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 21:59:59.929572 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5806f775-0b84-4e74-89db-0972c4d059c4 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 21:59:59.930292 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5806f775-0b84-4e74-89db-0972c4d059c4 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 21:59:59.931997 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5806f775-0b84-4e74-89db-0972c4d059c4 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 21:59:59.944494 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application [None req-5806f775-0b84-4e74-89db-0972c4d059c4 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 13 21:59:59.944494 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 21:59:59.944494 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 13 21:59:59.944494 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 13 21:59:59.944494 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 13 21:59:59.944494 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 13 21:59:59.944494 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 13 21:59:59.944494 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 13 21:59:59.944494 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 13 21:59:59.944494 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Jul 13 21:59:59.944494 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 13 21:59:59.944494 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Jul 13 21:59:59.944494 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 21:59:59.944494 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 21:59:59.944494 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 21:59:59.944494 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 21:59:59.944494 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 21:59:59.946888 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 21:59:59.946888 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 21:59:59.946888 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 21:59:59.946888 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 21:59:59.946888 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 21:59:59.946888 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 21:59:59.946888 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 13 21:59:59.946888 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 13 21:59:59.946888 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 13 21:59:59.946888 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 13 21:59:59.946888 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 21:59:59.946888 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 21:59:59.946888 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 21:59:59.946888 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 21:59:59.946888 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 21:59:59.946888 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 21:59:59.948882 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 21:59:59.948882 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application with Lock( Jul 13 21:59:59.948882 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 21:59:59.948882 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._enter() Jul 13 21:59:59.948882 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 21:59:59.948882 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 21:59:59.948882 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 21:59:59.948882 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.creator() Jul 13 21:59:59.948882 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 21:59:59.948882 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application created_value = creator( Jul 13 21:59:59.948882 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 13 21:59:59.948882 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 13 21:59:59.948882 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 13 21:59:59.948882 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Jul 13 21:59:59.958569 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 444/878] 10.222.0.52 () {56 vars in 1126 bytes} [Wed Jul 13 21:59:59 2022] GET /identity/v3/services/object-store => generated 93 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 13 21:59:59.958569 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5806f775-0b84-4e74-89db-0972c4d059c4 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 21:59:59.961541 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f39ac111-aea5-411b-a11b-9f07aa1000fb None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 21:59:59.999153 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406b0142-5cea-4017-bcac-a3ff48da1a72 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 21:59:59.999830 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406b0142-5cea-4017-bcac-a3ff48da1a72 None admin] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:00:00.000109 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406b0142-5cea-4017-bcac-a3ff48da1a72 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:00:00.000377 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-406b0142-5cea-4017-bcac-a3ff48da1a72 None admin] PATH_INFO: `/v3/services` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:00:00.001220 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406b0142-5cea-4017-bcac-a3ff48da1a72 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:00:00.002751 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-406b0142-5cea-4017-bcac-a3ff48da1a72 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:00:00.004189 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:00:00.026976 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 435/879] 10.222.0.52 () {60 vars in 1141 bytes} [Wed Jul 13 21:59:59 2022] POST /identity/v3/services => generated 269 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 22:00:00.064567 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5806f775-0b84-4e74-89db-0972c4d059c4 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:00:00.109846 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-850525a0-7140-4926-99fd-8ae333f63492 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:00:00.110527 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850525a0-7140-4926-99fd-8ae333f63492 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:00:00.110807 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850525a0-7140-4926-99fd-8ae333f63492 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:00:00.111085 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-850525a0-7140-4926-99fd-8ae333f63492 None admin] PATH_INFO: `/v3/services` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:00:00.111824 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850525a0-7140-4926-99fd-8ae333f63492 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 22:00:00.112190 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850525a0-7140-4926-99fd-8ae333f63492 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:00:00.113634 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-850525a0-7140-4926-99fd-8ae333f63492 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:00:00.124438 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 445/880] 10.222.0.52 () {56 vars in 1122 bytes} [Wed Jul 13 21:59:59 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 22:00:00.131712 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406b0142-5cea-4017-bcac-a3ff48da1a72 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:00:00.243384 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-406b0142-5cea-4017-bcac-a3ff48da1a72 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:00:00.278647 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e686b32-bd6d-43c6-aaad-925035faad74 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:00:00.279652 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:00:00.280120 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e686b32-bd6d-43c6-aaad-925035faad74 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:00:00.280575 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e686b32-bd6d-43c6-aaad-925035faad74 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:00:00.281163 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e686b32-bd6d-43c6-aaad-925035faad74 None admin] PATH_INFO: `/v3/services` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:00:00.282401 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e686b32-bd6d-43c6-aaad-925035faad74 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 22:00:00.282761 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e686b32-bd6d-43c6-aaad-925035faad74 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:00:00.283519 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ce34ff-88ed-4bea-8d7c-db23382b7480 None None] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:00:00.284014 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ce34ff-88ed-4bea-8d7c-db23382b7480 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:00:00.284375 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e686b32-bd6d-43c6-aaad-925035faad74 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:00:00.284724 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6ce34ff-88ed-4bea-8d7c-db23382b7480 None None] PATH_INFO: `/` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:00:00.287121 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 446/881] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jul 13 22:00:00 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 22:00:00.292069 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 436/882] 10.222.0.52 () {56 vars in 1122 bytes} [Wed Jul 13 22:00:00 2022] GET /identity/v3/services?type=object-store => generated 129 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 22:00:00.294755 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba63d4b3-e9a5-4e1e-b1e8-5f3e0cb12ba9 None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:00:00.295330 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba63d4b3-e9a5-4e1e-b1e8-5f3e0cb12ba9 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:00:00.295831 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba63d4b3-e9a5-4e1e-b1e8-5f3e0cb12ba9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:00:00.314882 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-ba63d4b3-e9a5-4e1e-b1e8-5f3e0cb12ba9 None None] MFA Rules not processed for user `cc99c93da5074cf5a48bc2aa9a92c5e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 22:00:00.316462 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-ba63d4b3-e9a5-4e1e-b1e8-5f3e0cb12ba9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:00:00.390576 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 447/883] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jul 13 22:00:00 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 13 22:00:00.415452 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e686b32-bd6d-43c6-aaad-925035faad74 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:00:00.523660 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e686b32-bd6d-43c6-aaad-925035faad74 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:00:00.562018 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb8d210-051f-439b-b3e0-0640c1b206c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:00:00.563164 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb8d210-051f-439b-b3e0-0640c1b206c0 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:00:00.563639 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb8d210-051f-439b-b3e0-0640c1b206c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:00:00.564122 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcb8d210-051f-439b-b3e0-0640c1b206c0 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:00:00.575204 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcb8d210-051f-439b-b3e0-0640c1b206c0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 22:00:00.583533 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application [None req-fcb8d210-051f-439b-b3e0-0640c1b206c0 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 13 22:00:00.583533 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 22:00:00.583533 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 22:00:00.583533 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 22:00:00.583533 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 22:00:00.583533 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 22:00:00.583533 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 22:00:00.583533 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 22:00:00.583533 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 22:00:00.583533 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 22:00:00.583533 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 22:00:00.583533 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 22:00:00.583533 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 22:00:00.583533 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 22:00:00.583533 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 22:00:00.583533 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 22:00:00.584469 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 22:00:00.584469 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 22:00:00.584469 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 22:00:00.584469 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 22:00:00.584469 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 22:00:00.584469 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 22:00:00.584469 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 22:00:00.584469 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 22:00:00.584469 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 22:00:00.584469 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 22:00:00.584469 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 22:00:00.584469 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application with Lock( Jul 13 22:00:00.584469 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 22:00:00.584469 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._enter() Jul 13 22:00:00.584469 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 22:00:00.584469 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 22:00:00.585313 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 22:00:00.585313 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.creator() Jul 13 22:00:00.585313 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 22:00:00.585313 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application created_value = creator( Jul 13 22:00:00.585313 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 22:00:00.585313 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 22:00:00.585313 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 22:00:00.585313 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 22:00:00.585313 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 22:00:00.585313 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 22:00:00.585313 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 13 22:00:00.585313 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Jul 13 22:00:00.588535 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 437/884] 10.222.0.52 () {56 vars in 1120 bytes} [Wed Jul 13 22:00:00 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 22:00:00.594727 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba63d4b3-e9a5-4e1e-b1e8-5f3e0cb12ba9 None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:00:00.634639 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:00:00.694632 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba63d4b3-e9a5-4e1e-b1e8-5f3e0cb12ba9 None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:00:00.733194 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823c00e2-2ba1-4c68-8c78-b2a2c578e534 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:00:00.733890 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823c00e2-2ba1-4c68-8c78-b2a2c578e534 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:00:00.734171 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823c00e2-2ba1-4c68-8c78-b2a2c578e534 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:00:00.734445 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823c00e2-2ba1-4c68-8c78-b2a2c578e534 None admin] PATH_INFO: `/v3/roles` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:00:00.735233 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-823c00e2-2ba1-4c68-8c78-b2a2c578e534 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 22:00:00.735542 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-823c00e2-2ba1-4c68-8c78-b2a2c578e534 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:00:00.736930 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-823c00e2-2ba1-4c68-8c78-b2a2c578e534 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:00:00.750343 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 448/885] 10.222.0.52 () {56 vars in 1115 bytes} [Wed Jul 13 22:00:00 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 22:00:00.755574 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb8d210-051f-439b-b3e0-0640c1b206c0 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:00:00.809773 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:00:00.841073 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcaa6e1b-aef8-4dd8-9e63-55a6a513c52d None None] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:00:00.841701 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcaa6e1b-aef8-4dd8-9e63-55a6a513c52d None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:00:00.842171 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcaa6e1b-aef8-4dd8-9e63-55a6a513c52d None None] PATH_INFO: `/` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:00:00.844027 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 449/886] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jul 13 22:00:00 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 13 22:00:00.848974 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf92173-2434-41fe-9455-afffe7f27622 None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:00:00.849487 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf92173-2434-41fe-9455-afffe7f27622 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:00:00.849819 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddf92173-2434-41fe-9455-afffe7f27622 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:00:00.864427 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcb8d210-051f-439b-b3e0-0640c1b206c0 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:00:00.870735 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-ddf92173-2434-41fe-9455-afffe7f27622 None None] MFA Rules not processed for user `cc99c93da5074cf5a48bc2aa9a92c5e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 22:00:00.872571 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-ddf92173-2434-41fe-9455-afffe7f27622 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:00:00.903715 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f91962b-1a53-43a6-9e82-f2bac0d45e4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:00:00.905311 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f91962b-1a53-43a6-9e82-f2bac0d45e4f None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:00:00.906047 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f91962b-1a53-43a6-9e82-f2bac0d45e4f None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:00:00.906577 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f91962b-1a53-43a6-9e82-f2bac0d45e4f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:00:00.919649 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f91962b-1a53-43a6-9e82-f2bac0d45e4f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:00:00.921794 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f91962b-1a53-43a6-9e82-f2bac0d45e4f None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:00:00.944406 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application [None req-7f91962b-1a53-43a6-9e82-f2bac0d45e4f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 22:00:00.944406 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 22:00:00.944406 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 22:00:00.944406 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 22:00:00.944406 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 22:00:00.944406 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 22:00:00.944406 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 22:00:00.944406 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 22:00:00.944406 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 22:00:00.944406 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 22:00:00.944406 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 22:00:00.944406 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 22:00:00.944406 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 22:00:00.944406 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 22:00:00.944406 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 22:00:00.945429 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 22:00:00.945429 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 22:00:00.945429 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 22:00:00.945429 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 22:00:00.945429 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 22:00:00.945429 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 22:00:00.945429 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 22:00:00.945429 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 22:00:00.945429 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application with Lock( Jul 13 22:00:00.945429 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 22:00:00.945429 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._enter() Jul 13 22:00:00.945429 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 22:00:00.945429 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 22:00:00.945429 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 22:00:00.945429 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.creator() Jul 13 22:00:00.945429 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 22:00:00.945429 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application created_value = creator( Jul 13 22:00:00.946463 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 22:00:00.946463 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 22:00:00.946463 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 22:00:00.946463 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Jul 13 22:00:00.954507 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 450/887] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jul 13 22:00:00 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 13 22:00:00.954643 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 438/888] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Jul 13 22:00:00 2022] GET /identity/v3/domains/Default => generated 87 bytes in 196 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 22:00:00.956879 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf92173-2434-41fe-9455-afffe7f27622 None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:00:00.974793 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f91962b-1a53-43a6-9e82-f2bac0d45e4f None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:00:00.994842 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:00:01.062792 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddf92173-2434-41fe-9455-afffe7f27622 None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:00:01.077124 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:00:01.079984 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f91962b-1a53-43a6-9e82-f2bac0d45e4f None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:00:01.103955 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7627c5ba-d106-48db-96b6-08d866df1cc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:00:01.105081 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7627c5ba-d106-48db-96b6-08d866df1cc7 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:00:01.105632 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7627c5ba-d106-48db-96b6-08d866df1cc7 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:00:01.106093 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7627c5ba-d106-48db-96b6-08d866df1cc7 None admin] PATH_INFO: `/v3/domains` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:00:01.107076 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7627c5ba-d106-48db-96b6-08d866df1cc7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 22:00:01.107680 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7627c5ba-d106-48db-96b6-08d866df1cc7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:00:01.109349 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7627c5ba-d106-48db-96b6-08d866df1cc7 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:00:01.115896 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085411d8-927f-4de8-8ec8-b64a06afac7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:00:01.116949 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085411d8-927f-4de8-8ec8-b64a06afac7d None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:00:01.116949 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085411d8-927f-4de8-8ec8-b64a06afac7d None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:00:01.117205 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-085411d8-927f-4de8-8ec8-b64a06afac7d None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:00:01.118086 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085411d8-927f-4de8-8ec8-b64a06afac7d None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:00:01.119684 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-085411d8-927f-4de8-8ec8-b64a06afac7d None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:00:01.125063 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 451/889] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Jul 13 22:00:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 22:00:01.130541 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7627c5ba-d106-48db-96b6-08d866df1cc7 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:00:01.136477 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application [None req-085411d8-927f-4de8-8ec8-b64a06afac7d None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 13 22:00:01.136477 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 22:00:01.136477 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 13 22:00:01.136477 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 13 22:00:01.136477 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 13 22:00:01.136477 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 13 22:00:01.136477 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 13 22:00:01.136477 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 13 22:00:01.136477 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 13 22:00:01.136477 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Jul 13 22:00:01.136477 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 13 22:00:01.136477 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Jul 13 22:00:01.136477 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 22:00:01.136477 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 22:00:01.136477 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 22:00:01.136477 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 22:00:01.136477 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 22:00:01.137504 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 22:00:01.137504 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 22:00:01.137504 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 22:00:01.137504 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 22:00:01.137504 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 22:00:01.137504 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 22:00:01.137504 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 13 22:00:01.137504 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 13 22:00:01.137504 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 13 22:00:01.137504 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 13 22:00:01.137504 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 22:00:01.137504 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 22:00:01.137504 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 22:00:01.137504 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 22:00:01.137504 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 22:00:01.137504 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 22:00:01.138348 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 22:00:01.138348 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application with Lock( Jul 13 22:00:01.138348 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 22:00:01.138348 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._enter() Jul 13 22:00:01.138348 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 22:00:01.138348 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 22:00:01.138348 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 22:00:01.138348 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.creator() Jul 13 22:00:01.138348 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 22:00:01.138348 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application created_value = creator( Jul 13 22:00:01.138348 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 13 22:00:01.138348 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 13 22:00:01.138348 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 13 22:00:01.138348 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Jul 13 22:00:01.142707 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 439/890] 10.222.0.52 () {56 vars in 1114 bytes} [Wed Jul 13 22:00:00 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 13 22:00:01.143599 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085411d8-927f-4de8-8ec8-b64a06afac7d None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:00:01.246795 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7627c5ba-d106-48db-96b6-08d866df1cc7 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:00:01.247257 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-085411d8-927f-4de8-8ec8-b64a06afac7d None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:00:01.254875 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:00:01.282139 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4738aa7-5f31-4f31-bfdb-be9c516da1b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:00:01.282950 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4738aa7-5f31-4f31-bfdb-be9c516da1b3 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:00:01.283230 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4738aa7-5f31-4f31-bfdb-be9c516da1b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:00:01.283501 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4738aa7-5f31-4f31-bfdb-be9c516da1b3 None admin] PATH_INFO: `/v3/services` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:00:01.284409 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4738aa7-5f31-4f31-bfdb-be9c516da1b3 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 22:00:01.284700 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4738aa7-5f31-4f31-bfdb-be9c516da1b3 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:00:01.285965 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77625086-82d7-4dea-8e03-ae73fe0e5854 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:00:01.286198 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4738aa7-5f31-4f31-bfdb-be9c516da1b3 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:00:01.287284 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77625086-82d7-4dea-8e03-ae73fe0e5854 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:00:01.287788 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77625086-82d7-4dea-8e03-ae73fe0e5854 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:00:01.288251 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77625086-82d7-4dea-8e03-ae73fe0e5854 None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:00:01.297833 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 440/891] 10.222.0.52 () {56 vars in 1114 bytes} [Wed Jul 13 22:00:01 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 22:00:01.298867 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4738aa7-5f31-4f31-bfdb-be9c516da1b3 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:00:01.299208 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77625086-82d7-4dea-8e03-ae73fe0e5854 None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:00:01.300869 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77625086-82d7-4dea-8e03-ae73fe0e5854 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:00:01.316600 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application [None req-77625086-82d7-4dea-8e03-ae73fe0e5854 None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 13 22:00:01.316600 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 22:00:01.316600 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 22:00:01.316600 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 22:00:01.316600 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 22:00:01.316600 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 22:00:01.316600 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 22:00:01.316600 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 22:00:01.316600 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 22:00:01.316600 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 22:00:01.316600 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 22:00:01.316600 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 22:00:01.316600 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 13 22:00:01.316600 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 13 22:00:01.316600 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 13 22:00:01.316600 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 13 22:00:01.317554 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 22:00:01.317554 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 22:00:01.317554 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 13 22:00:01.317554 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 22:00:01.317554 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 13 22:00:01.317554 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 22:00:01.317554 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 22:00:01.317554 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 22:00:01.317554 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 22:00:01.317554 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 22:00:01.317554 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 22:00:01.317554 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application with Lock( Jul 13 22:00:01.317554 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 22:00:01.317554 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._enter() Jul 13 22:00:01.317554 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 22:00:01.317554 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 22:00:01.317554 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 22:00:01.317554 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.creator() Jul 13 22:00:01.318543 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 22:00:01.318543 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application created_value = creator( Jul 13 22:00:01.318543 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 13 22:00:01.318543 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 13 22:00:01.318543 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 13 22:00:01.318543 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 13 22:00:01.318543 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 13 22:00:01.318543 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 13 22:00:01.318543 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 13 22:00:01.318543 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Jul 13 22:00:01.321125 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 452/892] 10.222.0.52 () {56 vars in 1117 bytes} [Wed Jul 13 22:00:01 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 22:00:01.327202 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77625086-82d7-4dea-8e03-ae73fe0e5854 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:00:01.346844 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:00:01.407603 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4738aa7-5f31-4f31-bfdb-be9c516da1b3 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:00:01.431508 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77625086-82d7-4dea-8e03-ae73fe0e5854 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:00:01.444246 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6e962d-9c2a-4de8-b969-fbeffe0c8ace None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:00:01.445188 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6e962d-9c2a-4de8-b969-fbeffe0c8ace None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:00:01.445759 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6e962d-9c2a-4de8-b969-fbeffe0c8ace None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:00:01.446257 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a6e962d-9c2a-4de8-b969-fbeffe0c8ace None admin] PATH_INFO: `/v3/services` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:00:01.447442 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6e962d-9c2a-4de8-b969-fbeffe0c8ace None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 22:00:01.447951 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6e962d-9c2a-4de8-b969-fbeffe0c8ace None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:00:01.449985 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a6e962d-9c2a-4de8-b969-fbeffe0c8ace None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:00:01.455754 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 441/893] 10.222.0.52 () {56 vars in 1114 bytes} [Wed Jul 13 22:00:01 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 22:00:01.461677 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6e962d-9c2a-4de8-b969-fbeffe0c8ace None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:00:01.474333 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5cdc009-93bc-48e2-878f-c182836e431f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:00:01.475727 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cdc009-93bc-48e2-878f-c182836e431f None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:00:01.476538 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cdc009-93bc-48e2-878f-c182836e431f None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:00:01.477404 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cdc009-93bc-48e2-878f-c182836e431f None admin] PATH_INFO: `/v3/users` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:00:01.478458 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5cdc009-93bc-48e2-878f-c182836e431f None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 22:00:01.479126 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5cdc009-93bc-48e2-878f-c182836e431f None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:00:01.481483 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5cdc009-93bc-48e2-878f-c182836e431f None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:00:01.504612 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 453/894] 10.222.0.52 () {56 vars in 1149 bytes} [Wed Jul 13 22:00:01 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 22:00:01.511945 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5cdc009-93bc-48e2-878f-c182836e431f None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:00:01.526040 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:00:01.579664 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:00:01.593813 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a6e962d-9c2a-4de8-b969-fbeffe0c8ace None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:00:01.639364 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6acd50ad-0acf-46a7-a656-b61c402f9157 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:00:01.640903 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acd50ad-0acf-46a7-a656-b61c402f9157 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:00:01.641613 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acd50ad-0acf-46a7-a656-b61c402f9157 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:00:01.642062 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6acd50ad-0acf-46a7-a656-b61c402f9157 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:00:01.643353 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6acd50ad-0acf-46a7-a656-b61c402f9157 None admin] RBAC: Adding query filter params (interface=public, service_id=7dad707752fb4c40af99c7e569ca422f, region_id=RegionOne) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 22:00:01.644217 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6acd50ad-0acf-46a7-a656-b61c402f9157 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:00:01.646296 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6acd50ad-0acf-46a7-a656-b61c402f9157 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:00:01.652555 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5cdc009-93bc-48e2-878f-c182836e431f None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:00:01.653962 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 442/895] 10.222.0.52 () {56 vars in 1251 bytes} [Wed Jul 13 22:00:01 2022] GET /identity/v3/endpoints?service_id=7dad707752fb4c40af99c7e569ca422f&interface=public®ion_id=RegionOne => generated 194 bytes in 193 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 22:00:01.659397 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6acd50ad-0acf-46a7-a656-b61c402f9157 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:00:01.696312 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ec2e30-ef9c-48a9-ac19-7362a86c6aa1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:00:01.697041 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ec2e30-ef9c-48a9-ac19-7362a86c6aa1 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:00:01.697277 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ec2e30-ef9c-48a9-ac19-7362a86c6aa1 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:00:01.697549 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ec2e30-ef9c-48a9-ac19-7362a86c6aa1 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:00:01.783982 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ec2e30-ef9c-48a9-ac19-7362a86c6aa1 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:00:01.785312 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6acd50ad-0acf-46a7-a656-b61c402f9157 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:00:01.785779 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7ec2e30-ef9c-48a9-ac19-7362a86c6aa1 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:00:01.796405 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:00:01.833709 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e291e9d0-9cf5-46bc-b1f2-5131969c7d29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:00:01.835223 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e291e9d0-9cf5-46bc-b1f2-5131969c7d29 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:00:01.835947 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e291e9d0-9cf5-46bc-b1f2-5131969c7d29 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:00:01.836530 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e291e9d0-9cf5-46bc-b1f2-5131969c7d29 None admin] PATH_INFO: `/v3/services` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:00:01.837660 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e291e9d0-9cf5-46bc-b1f2-5131969c7d29 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:00:01.839641 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e291e9d0-9cf5-46bc-b1f2-5131969c7d29 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:00:01.848365 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 443/896] 10.222.0.52 () {56 vars in 1087 bytes} [Wed Jul 13 22:00:01 2022] GET /identity/v3/services => generated 1268 bytes in 189 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 22:00:01.898080 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 454/897] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Jul 13 22:00:01 2022] GET /identity/v3/auth/tokens => generated 1362 bytes in 386 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 13 22:00:01.903952 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e291e9d0-9cf5-46bc-b1f2-5131969c7d29 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:00:01.929554 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:00:02.018726 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e291e9d0-9cf5-46bc-b1f2-5131969c7d29 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:00:02.054338 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-255069bb-21d6-4b2f-b98f-7207cc429ca0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:00:02.055507 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255069bb-21d6-4b2f-b98f-7207cc429ca0 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:00:02.055963 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255069bb-21d6-4b2f-b98f-7207cc429ca0 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:00:02.056438 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-255069bb-21d6-4b2f-b98f-7207cc429ca0 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:00:02.068152 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-255069bb-21d6-4b2f-b98f-7207cc429ca0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:00:02.069867 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-255069bb-21d6-4b2f-b98f-7207cc429ca0 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:00:02.088751 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application [None req-255069bb-21d6-4b2f-b98f-7207cc429ca0 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 22:00:02.088751 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 22:00:02.088751 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 22:00:02.088751 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 22:00:02.088751 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 22:00:02.088751 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 22:00:02.088751 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 22:00:02.088751 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 22:00:02.088751 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 22:00:02.088751 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 22:00:02.088751 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 22:00:02.088751 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 22:00:02.088751 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 22:00:02.088751 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 22:00:02.088751 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 22:00:02.089804 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 22:00:02.089804 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 22:00:02.089804 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 22:00:02.089804 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 22:00:02.089804 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 22:00:02.089804 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 22:00:02.089804 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 22:00:02.089804 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 22:00:02.089804 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application with Lock( Jul 13 22:00:02.089804 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 22:00:02.089804 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._enter() Jul 13 22:00:02.089804 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 22:00:02.089804 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 22:00:02.089804 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 22:00:02.089804 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.creator() Jul 13 22:00:02.089804 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 22:00:02.089804 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application created_value = creator( Jul 13 22:00:02.090740 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 22:00:02.090740 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 22:00:02.090740 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 22:00:02.090740 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Jul 13 22:00:02.092855 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 444/898] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Jul 13 22:00:01 2022] GET /identity/v3/domains/Default => generated 87 bytes in 189 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 13 22:00:02.098235 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ec2e30-ef9c-48a9-ac19-7362a86c6aa1 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:00:02.118179 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2870c9-7f42-4d77-ab38-d3bc0e126489 None None] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:00:02.118403 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2870c9-7f42-4d77-ab38-d3bc0e126489 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:00:02.118529 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c2870c9-7f42-4d77-ab38-d3bc0e126489 None None] PATH_INFO: `/` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:00:02.120173 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 445/899] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jul 13 22:00:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 13 22:00:02.125117 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fee156c-9937-4a92-b579-d6bd47ec10da None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:00:02.125401 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fee156c-9937-4a92-b579-d6bd47ec10da None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:00:02.125674 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fee156c-9937-4a92-b579-d6bd47ec10da None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:00:02.145807 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-9fee156c-9937-4a92-b579-d6bd47ec10da None None] MFA Rules not processed for user `cc99c93da5074cf5a48bc2aa9a92c5e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 22:00:02.147285 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-9fee156c-9937-4a92-b579-d6bd47ec10da None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:00:02.209365 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7ec2e30-ef9c-48a9-ac19-7362a86c6aa1 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:00:02.220936 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:00:02.230789 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 446/900] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jul 13 22:00:02 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 13 22:00:02.253541 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fee156c-9937-4a92-b579-d6bd47ec10da None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:00:02.256007 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:00:02.262709 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38893467-7073-4730-a9e4-ba1f78fafeff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:00:02.263391 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38893467-7073-4730-a9e4-ba1f78fafeff None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:00:02.263698 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38893467-7073-4730-a9e4-ba1f78fafeff None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:00:02.263976 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38893467-7073-4730-a9e4-ba1f78fafeff None admin] PATH_INFO: `/v3/domains` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:00:02.264707 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38893467-7073-4730-a9e4-ba1f78fafeff None admin] RBAC: Adding query filter params (name=Default) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 22:00:02.265009 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38893467-7073-4730-a9e4-ba1f78fafeff None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:00:02.266440 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38893467-7073-4730-a9e4-ba1f78fafeff None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:00:02.279348 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 455/901] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Jul 13 22:00:02 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 22:00:02.284995 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38893467-7073-4730-a9e4-ba1f78fafeff None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:00:02.368095 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fee156c-9937-4a92-b579-d6bd47ec10da None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:00:02.403146 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451f71b2-4056-4129-b57a-4d42eee047cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:00:02.404939 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451f71b2-4056-4129-b57a-4d42eee047cc None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:00:02.405610 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451f71b2-4056-4129-b57a-4d42eee047cc None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:00:02.406264 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451f71b2-4056-4129-b57a-4d42eee047cc None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:00:02.407243 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451f71b2-4056-4129-b57a-4d42eee047cc None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:00:02.408827 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451f71b2-4056-4129-b57a-4d42eee047cc None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:00:02.410193 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38893467-7073-4730-a9e4-ba1f78fafeff None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:00:02.421720 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application [None req-451f71b2-4056-4129-b57a-4d42eee047cc None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 13 22:00:02.421720 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 22:00:02.421720 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 13 22:00:02.421720 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 13 22:00:02.421720 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 13 22:00:02.421720 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 13 22:00:02.421720 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 13 22:00:02.421720 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 13 22:00:02.421720 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 13 22:00:02.421720 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Jul 13 22:00:02.421720 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 13 22:00:02.421720 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Jul 13 22:00:02.421720 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 22:00:02.421720 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 22:00:02.421720 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 22:00:02.421720 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 22:00:02.422739 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 22:00:02.422739 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 22:00:02.422739 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 22:00:02.422739 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 22:00:02.422739 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 22:00:02.422739 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 22:00:02.422739 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 22:00:02.422739 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 13 22:00:02.422739 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 13 22:00:02.422739 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 13 22:00:02.422739 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 13 22:00:02.422739 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 22:00:02.422739 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 22:00:02.422739 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 22:00:02.422739 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 22:00:02.422739 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 22:00:02.423572 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 22:00:02.423572 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 22:00:02.423572 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application with Lock( Jul 13 22:00:02.423572 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 22:00:02.423572 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._enter() Jul 13 22:00:02.423572 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 22:00:02.423572 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 22:00:02.423572 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 22:00:02.423572 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.creator() Jul 13 22:00:02.423572 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 22:00:02.423572 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application created_value = creator( Jul 13 22:00:02.423572 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 13 22:00:02.423572 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 13 22:00:02.423572 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 13 22:00:02.423572 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Jul 13 22:00:02.431177 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 447/902] 10.222.0.52 () {56 vars in 1132 bytes} [Wed Jul 13 22:00:02 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 22:00:02.432292 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451f71b2-4056-4129-b57a-4d42eee047cc None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:00:02.448116 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74dcc6e6-ad35-47b2-a975-d2c3005389fc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:00:02.449288 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74dcc6e6-ad35-47b2-a975-d2c3005389fc None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:00:02.449912 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74dcc6e6-ad35-47b2-a975-d2c3005389fc None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:00:02.450187 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74dcc6e6-ad35-47b2-a975-d2c3005389fc None admin] PATH_INFO: `/v3/projects/service` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:00:02.458604 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74dcc6e6-ad35-47b2-a975-d2c3005389fc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:00:02.460217 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74dcc6e6-ad35-47b2-a975-d2c3005389fc None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:00:02.478034 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application [None req-74dcc6e6-ad35-47b2-a975-d2c3005389fc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 13 22:00:02.478034 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 22:00:02.478034 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 22:00:02.478034 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 22:00:02.478034 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 22:00:02.478034 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 22:00:02.478034 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 22:00:02.478034 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 22:00:02.478034 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 22:00:02.478034 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 22:00:02.478034 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 22:00:02.478034 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 22:00:02.478034 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 13 22:00:02.478034 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 13 22:00:02.478034 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 13 22:00:02.479046 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 13 22:00:02.479046 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 22:00:02.479046 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 22:00:02.479046 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 22:00:02.479046 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 22:00:02.479046 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 22:00:02.479046 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 22:00:02.479046 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 22:00:02.479046 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application with Lock( Jul 13 22:00:02.479046 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 22:00:02.479046 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._enter() Jul 13 22:00:02.479046 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 22:00:02.479046 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 22:00:02.479046 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 22:00:02.479046 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.creator() Jul 13 22:00:02.479046 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 22:00:02.479046 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application created_value = creator( Jul 13 22:00:02.480011 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 13 22:00:02.480011 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 13 22:00:02.480011 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 13 22:00:02.480011 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 13 22:00:02.480011 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 13 22:00:02.480011 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 13 22:00:02.480011 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 13 22:00:02.480011 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Jul 13 22:00:02.482570 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 456/903] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Jul 13 22:00:02 2022] GET /identity/v3/projects/service => generated 88 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 22:00:02.484928 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74dcc6e6-ad35-47b2-a975-d2c3005389fc None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:00:02.514962 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:00:02.522229 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:00:02.556779 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451f71b2-4056-4129-b57a-4d42eee047cc None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:00:02.592771 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4073e034-4dee-4414-afbc-a169eafeb59e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:00:02.593620 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4073e034-4dee-4414-afbc-a169eafeb59e None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:00:02.593906 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4073e034-4dee-4414-afbc-a169eafeb59e None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:00:02.594172 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4073e034-4dee-4414-afbc-a169eafeb59e None admin] PATH_INFO: `/v3/services` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:00:02.594912 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4073e034-4dee-4414-afbc-a169eafeb59e None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 22:00:02.595215 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4073e034-4dee-4414-afbc-a169eafeb59e None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:00:02.596799 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4073e034-4dee-4414-afbc-a169eafeb59e None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:00:02.599068 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74dcc6e6-ad35-47b2-a975-d2c3005389fc None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:00:02.601852 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 448/904] 10.222.0.52 () {56 vars in 1126 bytes} [Wed Jul 13 22:00:02 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 22:00:02.603257 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8146b165-1c96-45bd-9a67-ab0c0d75269c None None] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:00:02.603540 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8146b165-1c96-45bd-9a67-ab0c0d75269c None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:00:02.603814 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8146b165-1c96-45bd-9a67-ab0c0d75269c None None] PATH_INFO: `/` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:00:02.609269 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 449/905] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jul 13 22:00:02 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 22:00:02.609269 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8146b165-1c96-45bd-9a67-ab0c0d75269c None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:00:02.640045 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5727f61-bd8f-43a4-84f0-89b29ad08dc1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:00:02.640635 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5727f61-bd8f-43a4-84f0-89b29ad08dc1 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:00:02.640911 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5727f61-bd8f-43a4-84f0-89b29ad08dc1 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:00:02.641181 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5727f61-bd8f-43a4-84f0-89b29ad08dc1 None admin] PATH_INFO: `/v3/projects` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:00:02.641967 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5727f61-bd8f-43a4-84f0-89b29ad08dc1 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 22:00:02.642275 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5727f61-bd8f-43a4-84f0-89b29ad08dc1 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:00:02.643741 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c5727f61-bd8f-43a4-84f0-89b29ad08dc1 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:00:02.655993 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 457/906] 10.222.0.52 () {56 vars in 1148 bytes} [Wed Jul 13 22:00:02 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 22:00:02.657253 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a53865-4a6e-4d80-bc32-0107deac6d0c None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:00:02.657533 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a53865-4a6e-4d80-bc32-0107deac6d0c None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:00:02.657802 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2a53865-4a6e-4d80-bc32-0107deac6d0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:00:02.674381 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-f2a53865-4a6e-4d80-bc32-0107deac6d0c None None] MFA Rules not processed for user `cc99c93da5074cf5a48bc2aa9a92c5e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 22:00:02.675592 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-f2a53865-4a6e-4d80-bc32-0107deac6d0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:00:02.723758 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8146b165-1c96-45bd-9a67-ab0c0d75269c None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:00:02.743871 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 458/907] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jul 13 22:00:02 2022] POST /identity/v3/auth/tokens => generated 1362 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 13 22:00:02.752066 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:00:02.755406 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a53865-4a6e-4d80-bc32-0107deac6d0c None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:00:02.758921 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82be4bc8-093b-4ee6-9fdf-ed7ff4682fb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:00:02.759863 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82be4bc8-093b-4ee6-9fdf-ed7ff4682fb7 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:00:02.760360 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82be4bc8-093b-4ee6-9fdf-ed7ff4682fb7 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:00:02.760854 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82be4bc8-093b-4ee6-9fdf-ed7ff4682fb7 None admin] PATH_INFO: `/v3/services` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:00:02.761731 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82be4bc8-093b-4ee6-9fdf-ed7ff4682fb7 None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 22:00:02.762242 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82be4bc8-093b-4ee6-9fdf-ed7ff4682fb7 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:00:02.763913 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82be4bc8-093b-4ee6-9fdf-ed7ff4682fb7 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:00:02.774428 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 450/908] 10.222.0.52 () {56 vars in 1126 bytes} [Wed Jul 13 22:00:02 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 22:00:02.775587 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82be4bc8-093b-4ee6-9fdf-ed7ff4682fb7 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:00:02.776892 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:00:02.863144 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2a53865-4a6e-4d80-bc32-0107deac6d0c None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:00:02.889985 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82be4bc8-093b-4ee6-9fdf-ed7ff4682fb7 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:00:02.898981 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530a7418-935b-470b-9d91-9e665f8b2ea2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:00:02.900167 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530a7418-935b-470b-9d91-9e665f8b2ea2 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:00:02.900450 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530a7418-935b-470b-9d91-9e665f8b2ea2 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:00:02.900726 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530a7418-935b-470b-9d91-9e665f8b2ea2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:00:02.901571 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530a7418-935b-470b-9d91-9e665f8b2ea2 None admin] RBAC: Adding query filter params (role.id=3987457f5fe64b568601d117c2c0ba9d, scope.project.id=417e3e2f1333481795a089bef2e24bea, user.id=b866bacdafae4db6a32714c07c56515f) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 22:00:02.901881 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530a7418-935b-470b-9d91-9e665f8b2ea2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:00:02.903345 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530a7418-935b-470b-9d91-9e665f8b2ea2 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:00:02.918521 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 459/909] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Jul 13 22:00:02 2022] GET /identity/v3/role_assignments?user.id=b866bacdafae4db6a32714c07c56515f&scope.project.id=417e3e2f1333481795a089bef2e24bea&role.id=3987457f5fe64b568601d117c2c0ba9d => generated 259 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 22:00:02.918521 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530a7418-935b-470b-9d91-9e665f8b2ea2 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:00:02.935016 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67e7d2d-bc5e-49e4-9fc2-47c38755d476 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:00:02.936381 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67e7d2d-bc5e-49e4-9fc2-47c38755d476 None admin] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:00:02.937456 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67e7d2d-bc5e-49e4-9fc2-47c38755d476 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:00:02.937728 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67e7d2d-bc5e-49e4-9fc2-47c38755d476 None admin] PATH_INFO: `/v3/services` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:00:02.940402 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67e7d2d-bc5e-49e4-9fc2-47c38755d476 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:00:02.942906 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67e7d2d-bc5e-49e4-9fc2-47c38755d476 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:00:02.970711 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 451/910] 10.222.0.52 () {60 vars in 1141 bytes} [Wed Jul 13 22:00:02 2022] POST /identity/v3/services => generated 241 bytes in 195 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 22:00:03.046602 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:00:03.069453 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530a7418-935b-470b-9d91-9e665f8b2ea2 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:00:03.118401 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60693e04-7b4c-4271-85e5-433109986991 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:00:03.119546 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60693e04-7b4c-4271-85e5-433109986991 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:00:03.120009 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60693e04-7b4c-4271-85e5-433109986991 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:00:03.120451 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60693e04-7b4c-4271-85e5-433109986991 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:00:03.121433 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60693e04-7b4c-4271-85e5-433109986991 None admin] RBAC: Adding query filter params (interface=public, service_id=7b49fb827b204eb19288eb48f08bd76a, region_id=RegionOne) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 22:00:03.121930 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60693e04-7b4c-4271-85e5-433109986991 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:00:03.123638 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60693e04-7b4c-4271-85e5-433109986991 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:00:03.133159 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 460/911] 10.222.0.52 () {56 vars in 1251 bytes} [Wed Jul 13 22:00:02 2022] GET /identity/v3/endpoints?service_id=7b49fb827b204eb19288eb48f08bd76a&interface=public®ion_id=RegionOne => generated 194 bytes in 223 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 22:00:03.138410 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67e7d2d-bc5e-49e4-9fc2-47c38755d476 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:00:03.200833 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:00:03.249728 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67e7d2d-bc5e-49e4-9fc2-47c38755d476 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:00:03.284643 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6e4f74-7ecd-4f14-b7cf-1eed4f079632 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:00:03.285362 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6e4f74-7ecd-4f14-b7cf-1eed4f079632 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:00:03.285845 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6e4f74-7ecd-4f14-b7cf-1eed4f079632 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:00:03.286133 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6e4f74-7ecd-4f14-b7cf-1eed4f079632 None admin] PATH_INFO: `/v3/services` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:00:03.286986 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b6e4f74-7ecd-4f14-b7cf-1eed4f079632 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:00:03.288647 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6b6e4f74-7ecd-4f14-b7cf-1eed4f079632 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:00:03.296707 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 452/912] 10.222.0.52 () {56 vars in 1087 bytes} [Wed Jul 13 22:00:03 2022] GET /identity/v3/services => generated 1497 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 13 22:00:03.941914 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6313f9-aa60-4856-a066-53897f532cfd None None] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:00:03.942463 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6313f9-aa60-4856-a066-53897f532cfd None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:00:03.942463 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed6313f9-aa60-4856-a066-53897f532cfd None None] PATH_INFO: `/` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:00:03.944088 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 461/913] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jul 13 22:00:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 22:00:03.950128 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d1dba6-885c-4216-90c7-37a532456e52 None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:00:03.950589 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d1dba6-885c-4216-90c7-37a532456e52 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:00:03.950766 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d1dba6-885c-4216-90c7-37a532456e52 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:00:03.967631 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-a7d1dba6-885c-4216-90c7-37a532456e52 None None] MFA Rules not processed for user `cc99c93da5074cf5a48bc2aa9a92c5e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 22:00:03.968808 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-a7d1dba6-885c-4216-90c7-37a532456e52 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:00:04.041753 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 453/914] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jul 13 22:00:03 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 13 22:00:04.064524 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6313f9-aa60-4856-a066-53897f532cfd None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:00:04.171092 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed6313f9-aa60-4856-a066-53897f532cfd None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:00:04.207177 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e8a392-204f-4f81-b4a4-4ebabd635121 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:00:04.208242 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e8a392-204f-4f81-b4a4-4ebabd635121 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:00:04.208788 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e8a392-204f-4f81-b4a4-4ebabd635121 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:00:04.209207 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64e8a392-204f-4f81-b4a4-4ebabd635121 None admin] PATH_INFO: `/v3/services/volumev3` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:00:04.210027 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e8a392-204f-4f81-b4a4-4ebabd635121 None admin] RBAC: Authorizing `identity:get_service(service_id=volumev3)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:00:04.211715 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64e8a392-204f-4f81-b4a4-4ebabd635121 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:00:04.225280 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application [None req-64e8a392-204f-4f81-b4a4-4ebabd635121 None admin] Could not find service: volumev3.: keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 13 22:00:04.225280 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 22:00:04.225280 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 13 22:00:04.225280 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 13 22:00:04.225280 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 13 22:00:04.225280 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 13 22:00:04.225280 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 13 22:00:04.225280 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 13 22:00:04.225280 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 13 22:00:04.225280 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Jul 13 22:00:04.225280 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 13 22:00:04.225280 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Jul 13 22:00:04.225280 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 22:00:04.225280 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 22:00:04.225280 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 22:00:04.225280 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 22:00:04.225280 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 22:00:04.226611 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 22:00:04.226611 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 22:00:04.226611 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 22:00:04.226611 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 22:00:04.226611 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 22:00:04.226611 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 22:00:04.226611 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 13 22:00:04.226611 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 13 22:00:04.226611 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 13 22:00:04.226611 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 13 22:00:04.226611 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 22:00:04.226611 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 22:00:04.226611 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 22:00:04.226611 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 22:00:04.226611 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 22:00:04.226611 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 22:00:04.227530 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 22:00:04.227530 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application with Lock( Jul 13 22:00:04.227530 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 22:00:04.227530 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._enter() Jul 13 22:00:04.227530 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 22:00:04.227530 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 22:00:04.227530 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 22:00:04.227530 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.creator() Jul 13 22:00:04.227530 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 22:00:04.227530 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application created_value = creator( Jul 13 22:00:04.227530 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 13 22:00:04.227530 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 13 22:00:04.227530 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: volumev3. Jul 13 22:00:04.227530 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Jul 13 22:00:04.228854 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 462/915] 10.222.0.52 () {56 vars in 1114 bytes} [Wed Jul 13 22:00:04 2022] GET /identity/v3/services/volumev3 => generated 89 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 22:00:04.235804 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d1dba6-885c-4216-90c7-37a532456e52 None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:00:04.284699 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:00:04.335734 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d1dba6-885c-4216-90c7-37a532456e52 None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:00:04.369721 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2813d83d-ebf7-4e6e-832e-692715880a03 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:00:04.370392 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2813d83d-ebf7-4e6e-832e-692715880a03 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:00:04.370721 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2813d83d-ebf7-4e6e-832e-692715880a03 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:00:04.370989 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2813d83d-ebf7-4e6e-832e-692715880a03 None admin] PATH_INFO: `/v3/services` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:00:04.371702 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2813d83d-ebf7-4e6e-832e-692715880a03 None admin] RBAC: Adding query filter params (name=volumev3) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 22:00:04.372002 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2813d83d-ebf7-4e6e-832e-692715880a03 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:00:04.373366 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2813d83d-ebf7-4e6e-832e-692715880a03 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:00:04.378608 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 454/916] 10.222.0.52 () {56 vars in 1114 bytes} [Wed Jul 13 22:00:04 2022] GET /identity/v3/services?name=volumev3 => generated 125 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 22:00:04.384187 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e8a392-204f-4f81-b4a4-4ebabd635121 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:00:04.394051 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:00:04.492680 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64e8a392-204f-4f81-b4a4-4ebabd635121 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:00:04.527988 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a51abd48-169a-4386-b470-3014a6514151 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:00:04.529003 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51abd48-169a-4386-b470-3014a6514151 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:00:04.529519 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51abd48-169a-4386-b470-3014a6514151 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:00:04.529990 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a51abd48-169a-4386-b470-3014a6514151 None admin] PATH_INFO: `/v3/services` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:00:04.530937 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51abd48-169a-4386-b470-3014a6514151 None admin] RBAC: Adding query filter params (type=volumev3) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 22:00:04.531441 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51abd48-169a-4386-b470-3014a6514151 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:00:04.533018 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a51abd48-169a-4386-b470-3014a6514151 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:00:04.538689 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 463/917] 10.222.0.52 () {56 vars in 1114 bytes} [Wed Jul 13 22:00:04 2022] GET /identity/v3/services?type=volumev3 => generated 362 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 22:00:04.549788 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2813d83d-ebf7-4e6e-832e-692715880a03 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:00:04.648442 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2813d83d-ebf7-4e6e-832e-692715880a03 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:00:04.661999 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:00:04.684357 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-349777f0-bb64-4c59-96e8-f781f7e05c36 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:00:04.685024 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349777f0-bb64-4c59-96e8-f781f7e05c36 None admin] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:00:04.685302 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349777f0-bb64-4c59-96e8-f781f7e05c36 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:00:04.685572 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-349777f0-bb64-4c59-96e8-f781f7e05c36 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:00:04.686320 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-349777f0-bb64-4c59-96e8-f781f7e05c36 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:00:04.687817 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-349777f0-bb64-4c59-96e8-f781f7e05c36 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:00:04.713319 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 455/918] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Jul 13 22:00:04 2022] POST /identity/v3/endpoints => generated 346 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 22:00:05.009212 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cbac7a-2a2d-4c9f-8562-e89047eb007f None None] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:00:05.009757 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cbac7a-2a2d-4c9f-8562-e89047eb007f None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:00:05.009757 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cbac7a-2a2d-4c9f-8562-e89047eb007f None None] PATH_INFO: `/` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:00:05.011427 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 464/919] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jul 13 22:00:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 22:00:05.016838 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0ee8e7-b404-4c56-890a-ada0a8eb33ed None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:00:05.017267 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0ee8e7-b404-4c56-890a-ada0a8eb33ed None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:00:05.017500 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a0ee8e7-b404-4c56-890a-ada0a8eb33ed None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:00:05.034308 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-0a0ee8e7-b404-4c56-890a-ada0a8eb33ed None None] MFA Rules not processed for user `cc99c93da5074cf5a48bc2aa9a92c5e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 22:00:05.035539 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-0a0ee8e7-b404-4c56-890a-ada0a8eb33ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:00:05.110163 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 456/920] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jul 13 22:00:05 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 13 22:00:05.132674 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5cbac7a-2a2d-4c9f-8562-e89047eb007f None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:00:05.161569 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c7f635-4fc4-4f46-990e-8aba0a89e17d None None] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:00:05.161883 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c7f635-4fc4-4f46-990e-8aba0a89e17d None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:00:05.162118 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c7f635-4fc4-4f46-990e-8aba0a89e17d None None] PATH_INFO: `/` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:00:05.163806 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 457/921] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jul 13 22:00:05 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 22:00:05.170019 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4515f730-3855-4ca0-8f70-ba979c96c204 None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:00:05.170460 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4515f730-3855-4ca0-8f70-ba979c96c204 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:00:05.170931 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4515f730-3855-4ca0-8f70-ba979c96c204 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:00:05.190125 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-4515f730-3855-4ca0-8f70-ba979c96c204 None None] MFA Rules not processed for user `cc99c93da5074cf5a48bc2aa9a92c5e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 22:00:05.191621 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-4515f730-3855-4ca0-8f70-ba979c96c204 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:00:05.218618 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:00:05.261330 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5cbac7a-2a2d-4c9f-8562-e89047eb007f None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:00:05.288327 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 458/922] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jul 13 22:00:05 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 13 22:00:05.305835 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac178043-0871-4df4-b30f-2c7c468d4779 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:00:05.306763 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac178043-0871-4df4-b30f-2c7c468d4779 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:00:05.307044 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac178043-0871-4df4-b30f-2c7c468d4779 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:00:05.307317 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac178043-0871-4df4-b30f-2c7c468d4779 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:00:05.309148 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4515f730-3855-4ca0-8f70-ba979c96c204 None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:00:05.324698 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac178043-0871-4df4-b30f-2c7c468d4779 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 22:00:05.335658 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application [None req-ac178043-0871-4df4-b30f-2c7c468d4779 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 13 22:00:05.335658 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 22:00:05.335658 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 22:00:05.335658 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 22:00:05.335658 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 22:00:05.335658 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 22:00:05.335658 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 22:00:05.335658 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 22:00:05.335658 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 22:00:05.335658 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 22:00:05.335658 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 22:00:05.335658 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 22:00:05.335658 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 22:00:05.335658 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 22:00:05.335658 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 22:00:05.335658 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 22:00:05.336825 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 22:00:05.336825 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 22:00:05.336825 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 22:00:05.336825 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 22:00:05.336825 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 22:00:05.336825 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 22:00:05.336825 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 22:00:05.336825 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 22:00:05.336825 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 22:00:05.336825 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 22:00:05.336825 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 22:00:05.336825 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application with Lock( Jul 13 22:00:05.336825 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 22:00:05.336825 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._enter() Jul 13 22:00:05.336825 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 22:00:05.336825 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 22:00:05.339150 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 22:00:05.339150 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.creator() Jul 13 22:00:05.339150 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 22:00:05.339150 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application created_value = creator( Jul 13 22:00:05.339150 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 22:00:05.339150 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 22:00:05.339150 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 22:00:05.339150 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 22:00:05.339150 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 22:00:05.339150 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 22:00:05.339150 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 13 22:00:05.339150 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Jul 13 22:00:05.339150 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 465/923] 10.222.0.52 () {56 vars in 1120 bytes} [Wed Jul 13 22:00:05 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 22:00:05.343216 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac178043-0871-4df4-b30f-2c7c468d4779 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:00:05.345208 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:00:05.425867 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4515f730-3855-4ca0-8f70-ba979c96c204 None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:00:05.445311 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac178043-0871-4df4-b30f-2c7c468d4779 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:00:05.462569 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f85cb6b-335e-47df-8dfd-541921adac94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:00:05.463897 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f85cb6b-335e-47df-8dfd-541921adac94 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:00:05.464352 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f85cb6b-335e-47df-8dfd-541921adac94 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:00:05.464794 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f85cb6b-335e-47df-8dfd-541921adac94 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:00:05.465656 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f85cb6b-335e-47df-8dfd-541921adac94 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:00:05.467462 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f85cb6b-335e-47df-8dfd-541921adac94 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:00:05.475738 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:00:05.481122 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application [None req-8f85cb6b-335e-47df-8dfd-541921adac94 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 13 22:00:05.481122 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 22:00:05.481122 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 13 22:00:05.481122 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 13 22:00:05.481122 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 13 22:00:05.481122 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 13 22:00:05.481122 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 13 22:00:05.481122 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 13 22:00:05.481122 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 13 22:00:05.481122 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Jul 13 22:00:05.481122 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 13 22:00:05.481122 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Jul 13 22:00:05.481122 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 22:00:05.481122 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 22:00:05.481122 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 22:00:05.481122 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 22:00:05.481122 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 22:00:05.482216 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 22:00:05.482216 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 22:00:05.482216 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 22:00:05.482216 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 22:00:05.482216 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 22:00:05.482216 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 22:00:05.482216 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 13 22:00:05.482216 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 13 22:00:05.482216 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 13 22:00:05.482216 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 13 22:00:05.482216 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 22:00:05.482216 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 22:00:05.482216 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 22:00:05.482216 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 22:00:05.482216 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 22:00:05.482216 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 22:00:05.483302 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 22:00:05.483302 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application with Lock( Jul 13 22:00:05.483302 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 22:00:05.483302 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._enter() Jul 13 22:00:05.483302 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 22:00:05.483302 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 22:00:05.483302 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 22:00:05.483302 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.creator() Jul 13 22:00:05.483302 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 22:00:05.483302 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application created_value = creator( Jul 13 22:00:05.483302 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 13 22:00:05.483302 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 13 22:00:05.483302 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 13 22:00:05.483302 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Jul 13 22:00:05.483302 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 459/924] 10.222.0.52 () {56 vars in 1126 bytes} [Wed Jul 13 22:00:05 2022] GET /identity/v3/services/object-store => generated 93 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 13 22:00:05.484111 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42d4f1c-48bf-402e-98d1-c5742a65c0fc None None] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:00:05.484111 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42d4f1c-48bf-402e-98d1-c5742a65c0fc None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:00:05.484111 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b42d4f1c-48bf-402e-98d1-c5742a65c0fc None None] PATH_INFO: `/` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:00:05.485527 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 460/925] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jul 13 22:00:05 2022] GET /identity => generated 270 bytes in 2 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 22:00:05.488496 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42d4f1c-48bf-402e-98d1-c5742a65c0fc None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:00:05.490179 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cf1b4b3-cad2-472a-8dd1-924b426e8c78 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:00:05.490966 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf1b4b3-cad2-472a-8dd1-924b426e8c78 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:00:05.491244 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf1b4b3-cad2-472a-8dd1-924b426e8c78 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:00:05.491617 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cf1b4b3-cad2-472a-8dd1-924b426e8c78 None admin] PATH_INFO: `/v3/roles` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:00:05.492510 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf1b4b3-cad2-472a-8dd1-924b426e8c78 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 22:00:05.492822 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf1b4b3-cad2-472a-8dd1-924b426e8c78 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:00:05.494210 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cf1b4b3-cad2-472a-8dd1-924b426e8c78 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:00:05.507737 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 466/926] 10.222.0.52 () {56 vars in 1115 bytes} [Wed Jul 13 22:00:05 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 22:00:05.507737 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547d9541-0bd5-4f4f-be41-7385bd52e64c None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:00:05.507737 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547d9541-0bd5-4f4f-be41-7385bd52e64c None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:00:05.507737 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-547d9541-0bd5-4f4f-be41-7385bd52e64c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:00:05.523023 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-547d9541-0bd5-4f4f-be41-7385bd52e64c None None] MFA Rules not processed for user `cc99c93da5074cf5a48bc2aa9a92c5e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 22:00:05.524159 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-547d9541-0bd5-4f4f-be41-7385bd52e64c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:00:05.592858 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:00:05.614517 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 467/927] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jul 13 22:00:05 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 13 22:00:05.614517 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547d9541-0bd5-4f4f-be41-7385bd52e64c None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:00:05.616648 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b42d4f1c-48bf-402e-98d1-c5742a65c0fc None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:00:05.656672 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b970f1d6-4c82-4079-b048-ebe3ace3ee9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:00:05.657342 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b970f1d6-4c82-4079-b048-ebe3ace3ee9b None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:00:05.657618 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b970f1d6-4c82-4079-b048-ebe3ace3ee9b None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:00:05.657904 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b970f1d6-4c82-4079-b048-ebe3ace3ee9b None admin] PATH_INFO: `/v3/services` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:00:05.658655 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b970f1d6-4c82-4079-b048-ebe3ace3ee9b None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 22:00:05.658959 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b970f1d6-4c82-4079-b048-ebe3ace3ee9b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:00:05.660335 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b970f1d6-4c82-4079-b048-ebe3ace3ee9b None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:00:05.665142 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 461/928] 10.222.0.52 () {56 vars in 1122 bytes} [Wed Jul 13 22:00:05 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 177 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 22:00:05.667696 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b970f1d6-4c82-4079-b048-ebe3ace3ee9b None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:00:05.724678 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-547d9541-0bd5-4f4f-be41-7385bd52e64c None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:00:05.748235 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:00:05.760955 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c19457-6892-42f2-82eb-5d21efc4a746 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:00:05.762132 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c19457-6892-42f2-82eb-5d21efc4a746 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:00:05.762411 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c19457-6892-42f2-82eb-5d21efc4a746 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:00:05.762703 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4c19457-6892-42f2-82eb-5d21efc4a746 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:00:05.773016 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c19457-6892-42f2-82eb-5d21efc4a746 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:00:05.774953 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4c19457-6892-42f2-82eb-5d21efc4a746 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:00:05.782968 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b970f1d6-4c82-4079-b048-ebe3ace3ee9b None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:00:05.793426 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application [None req-f4c19457-6892-42f2-82eb-5d21efc4a746 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 22:00:05.793426 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 22:00:05.793426 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 22:00:05.793426 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 22:00:05.793426 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 22:00:05.793426 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 22:00:05.793426 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 22:00:05.793426 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 22:00:05.793426 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 22:00:05.793426 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 22:00:05.793426 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 22:00:05.793426 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 22:00:05.793426 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 22:00:05.793426 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 22:00:05.793426 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 22:00:05.794405 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 22:00:05.794405 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 22:00:05.794405 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 22:00:05.794405 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 22:00:05.794405 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 22:00:05.794405 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 22:00:05.794405 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 22:00:05.794405 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 22:00:05.794405 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application with Lock( Jul 13 22:00:05.794405 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 22:00:05.794405 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._enter() Jul 13 22:00:05.794405 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 22:00:05.794405 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 22:00:05.794405 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 22:00:05.794405 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.creator() Jul 13 22:00:05.794405 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 22:00:05.794405 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application created_value = creator( Jul 13 22:00:05.795443 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 22:00:05.795443 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 22:00:05.795443 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 22:00:05.795443 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Jul 13 22:00:05.795443 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 468/929] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Jul 13 22:00:05 2022] GET /identity/v3/domains/Default => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 22:00:05.795762 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c19457-6892-42f2-82eb-5d21efc4a746 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:00:05.825007 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f301b2a-3d02-40da-8179-42c958590dcc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:00:05.825836 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f301b2a-3d02-40da-8179-42c958590dcc None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:00:05.826117 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f301b2a-3d02-40da-8179-42c958590dcc None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:00:05.826602 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f301b2a-3d02-40da-8179-42c958590dcc None admin] PATH_INFO: `/v3/services/compute_legacy` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:00:05.827084 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f301b2a-3d02-40da-8179-42c958590dcc None admin] RBAC: Authorizing `identity:get_service(service_id=compute_legacy)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:00:05.828457 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f301b2a-3d02-40da-8179-42c958590dcc None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:00:05.841041 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application [None req-0f301b2a-3d02-40da-8179-42c958590dcc None admin] Could not find service: compute_legacy.: keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 13 22:00:05.841041 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 22:00:05.841041 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 13 22:00:05.841041 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 13 22:00:05.841041 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 13 22:00:05.841041 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 13 22:00:05.841041 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 13 22:00:05.841041 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 13 22:00:05.841041 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 13 22:00:05.841041 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Jul 13 22:00:05.841041 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 13 22:00:05.841041 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Jul 13 22:00:05.841041 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 22:00:05.841041 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 22:00:05.841041 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 22:00:05.841041 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 22:00:05.843024 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 22:00:05.843024 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 22:00:05.843024 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 22:00:05.843024 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 22:00:05.843024 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 22:00:05.843024 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 22:00:05.843024 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 22:00:05.843024 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 13 22:00:05.843024 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 13 22:00:05.843024 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 13 22:00:05.843024 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 13 22:00:05.843024 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 22:00:05.843024 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 22:00:05.843024 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 22:00:05.843024 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 22:00:05.843024 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 22:00:05.843907 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 22:00:05.843907 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 22:00:05.843907 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application with Lock( Jul 13 22:00:05.843907 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 22:00:05.843907 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._enter() Jul 13 22:00:05.843907 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 22:00:05.843907 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 22:00:05.843907 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 22:00:05.843907 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.creator() Jul 13 22:00:05.843907 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 22:00:05.843907 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application created_value = creator( Jul 13 22:00:05.843907 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 13 22:00:05.843907 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 13 22:00:05.843907 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute_legacy. Jul 13 22:00:05.843907 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Jul 13 22:00:05.843907 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 462/930] 10.222.0.52 () {56 vars in 1132 bytes} [Wed Jul 13 22:00:05 2022] GET /identity/v3/services/compute_legacy => generated 95 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 22:00:05.847757 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f301b2a-3d02-40da-8179-42c958590dcc None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:00:05.868021 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:00:05.906656 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4c19457-6892-42f2-82eb-5d21efc4a746 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:00:05.942633 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8225d1f-5b0b-4381-950e-b947d4847366 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:00:05.943320 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8225d1f-5b0b-4381-950e-b947d4847366 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:00:05.943601 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8225d1f-5b0b-4381-950e-b947d4847366 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:00:05.943873 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a8225d1f-5b0b-4381-950e-b947d4847366 None admin] PATH_INFO: `/v3/services` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:00:05.944589 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8225d1f-5b0b-4381-950e-b947d4847366 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 22:00:05.944894 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8225d1f-5b0b-4381-950e-b947d4847366 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:00:05.946614 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a8225d1f-5b0b-4381-950e-b947d4847366 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:00:05.956646 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 469/931] 10.222.0.52 () {56 vars in 1122 bytes} [Wed Jul 13 22:00:05 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 22:00:05.956646 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8225d1f-5b0b-4381-950e-b947d4847366 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:00:05.963127 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f301b2a-3d02-40da-8179-42c958590dcc None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:00:05.999106 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:00:06.000877 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e00585-29d8-4919-92e9-7d1fcffbd9e1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:00:06.002005 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e00585-29d8-4919-92e9-7d1fcffbd9e1 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:00:06.002286 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e00585-29d8-4919-92e9-7d1fcffbd9e1 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:00:06.002877 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72e00585-29d8-4919-92e9-7d1fcffbd9e1 None admin] PATH_INFO: `/v3/domains` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:00:06.003612 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e00585-29d8-4919-92e9-7d1fcffbd9e1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 22:00:06.003911 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e00585-29d8-4919-92e9-7d1fcffbd9e1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:00:06.005308 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72e00585-29d8-4919-92e9-7d1fcffbd9e1 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:00:06.018419 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 463/932] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Jul 13 22:00:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 22:00:06.019265 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e00585-29d8-4919-92e9-7d1fcffbd9e1 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:00:06.066859 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a8225d1f-5b0b-4381-950e-b947d4847366 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:00:06.101515 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2671661-ea83-4a47-bd41-22cce031343a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:00:06.102260 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2671661-ea83-4a47-bd41-22cce031343a None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:00:06.102579 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2671661-ea83-4a47-bd41-22cce031343a None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:00:06.102839 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2671661-ea83-4a47-bd41-22cce031343a None admin] PATH_INFO: `/v3/services` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:00:06.103398 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:00:06.103532 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2671661-ea83-4a47-bd41-22cce031343a None admin] RBAC: Adding query filter params (name=compute_legacy) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 22:00:06.103841 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2671661-ea83-4a47-bd41-22cce031343a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:00:06.105331 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2671661-ea83-4a47-bd41-22cce031343a None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:00:06.114682 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 470/933] 10.222.0.52 () {56 vars in 1126 bytes} [Wed Jul 13 22:00:05 2022] GET /identity/v3/services?name=compute_legacy => generated 131 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 22:00:06.114682 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2671661-ea83-4a47-bd41-22cce031343a None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:00:06.135626 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72e00585-29d8-4919-92e9-7d1fcffbd9e1 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:00:06.177361 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3fae57-490c-473a-a1af-43d7819dd635 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:00:06.178018 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3fae57-490c-473a-a1af-43d7819dd635 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:00:06.178281 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3fae57-490c-473a-a1af-43d7819dd635 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:00:06.178578 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c3fae57-490c-473a-a1af-43d7819dd635 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:00:06.179373 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3fae57-490c-473a-a1af-43d7819dd635 None admin] RBAC: Adding query filter params (interface=public, service_id=1b863c3a98f24d66a34f0bde03ae23a4, region_id=RegionOne) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 22:00:06.179681 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3fae57-490c-473a-a1af-43d7819dd635 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:00:06.181658 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c3fae57-490c-473a-a1af-43d7819dd635 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:00:06.191300 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 464/934] 10.222.0.52 () {56 vars in 1251 bytes} [Wed Jul 13 22:00:06 2022] GET /identity/v3/endpoints?service_id=1b863c3a98f24d66a34f0bde03ae23a4&interface=public®ion_id=RegionOne => generated 194 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 22:00:06.191300 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3fae57-490c-473a-a1af-43d7819dd635 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:00:06.219239 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2671661-ea83-4a47-bd41-22cce031343a None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:00:06.220971 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:00:06.256558 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73a9713-c84f-4f9d-8a82-f28d9b601cee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:00:06.257439 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73a9713-c84f-4f9d-8a82-f28d9b601cee None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:00:06.257721 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73a9713-c84f-4f9d-8a82-f28d9b601cee None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:00:06.257995 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73a9713-c84f-4f9d-8a82-f28d9b601cee None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:00:06.265851 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d73a9713-c84f-4f9d-8a82-f28d9b601cee None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:00:06.267400 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d73a9713-c84f-4f9d-8a82-f28d9b601cee None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:00:06.282410 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application [None req-d73a9713-c84f-4f9d-8a82-f28d9b601cee None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 13 22:00:06.282410 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 22:00:06.282410 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 22:00:06.282410 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 22:00:06.282410 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 22:00:06.282410 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 22:00:06.282410 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 22:00:06.282410 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 22:00:06.282410 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 22:00:06.282410 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 22:00:06.282410 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 22:00:06.282410 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 22:00:06.282410 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 13 22:00:06.282410 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 13 22:00:06.282410 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 13 22:00:06.282410 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 13 22:00:06.283619 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 22:00:06.283619 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 22:00:06.283619 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 13 22:00:06.283619 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 22:00:06.283619 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 13 22:00:06.283619 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 22:00:06.283619 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 22:00:06.283619 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 22:00:06.283619 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 22:00:06.283619 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 22:00:06.283619 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 22:00:06.283619 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application with Lock( Jul 13 22:00:06.283619 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 22:00:06.283619 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._enter() Jul 13 22:00:06.283619 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 22:00:06.283619 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 22:00:06.283619 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 22:00:06.283619 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.creator() Jul 13 22:00:06.284576 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 22:00:06.284576 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application created_value = creator( Jul 13 22:00:06.284576 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 13 22:00:06.284576 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 13 22:00:06.284576 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 13 22:00:06.284576 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 13 22:00:06.284576 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 13 22:00:06.284576 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 13 22:00:06.284576 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 13 22:00:06.284576 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Jul 13 22:00:06.284576 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 471/935] 10.222.0.52 () {56 vars in 1117 bytes} [Wed Jul 13 22:00:06 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 22:00:06.284576 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73a9713-c84f-4f9d-8a82-f28d9b601cee None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:00:06.300234 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c3fae57-490c-473a-a1af-43d7819dd635 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:00:06.339826 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16e1a05-327d-4844-b638-f4080494b10c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:00:06.340443 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16e1a05-327d-4844-b638-f4080494b10c None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:00:06.340723 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16e1a05-327d-4844-b638-f4080494b10c None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:00:06.341235 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a16e1a05-327d-4844-b638-f4080494b10c None admin] PATH_INFO: `/v3/services` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:00:06.341701 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16e1a05-327d-4844-b638-f4080494b10c None admin] RBAC: Adding query filter params (type=compute_legacy) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 22:00:06.342006 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16e1a05-327d-4844-b638-f4080494b10c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:00:06.343424 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a16e1a05-327d-4844-b638-f4080494b10c None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:00:06.348933 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 465/936] 10.222.0.52 () {56 vars in 1126 bytes} [Wed Jul 13 22:00:06 2022] GET /identity/v3/services?type=compute_legacy => generated 386 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 22:00:06.349717 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16e1a05-327d-4844-b638-f4080494b10c None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:00:06.353946 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:00:06.404972 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73a9713-c84f-4f9d-8a82-f28d9b601cee None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:00:06.443471 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06dbf5ae-9955-41b8-bbc0-0a3e2d6fb7c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:00:06.444147 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06dbf5ae-9955-41b8-bbc0-0a3e2d6fb7c1 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:00:06.444425 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06dbf5ae-9955-41b8-bbc0-0a3e2d6fb7c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:00:06.444697 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06dbf5ae-9955-41b8-bbc0-0a3e2d6fb7c1 None admin] PATH_INFO: `/v3/services` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:00:06.445860 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06dbf5ae-9955-41b8-bbc0-0a3e2d6fb7c1 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:00:06.447276 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06dbf5ae-9955-41b8-bbc0-0a3e2d6fb7c1 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:00:06.454082 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 472/937] 10.222.0.52 () {56 vars in 1087 bytes} [Wed Jul 13 22:00:06 2022] GET /identity/v3/services => generated 1497 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 22:00:06.454868 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06dbf5ae-9955-41b8-bbc0-0a3e2d6fb7c1 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:00:06.468338 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a16e1a05-327d-4844-b638-f4080494b10c None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:00:06.494473 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:00:06.520731 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a2c311-51ea-41b4-af13-8aa8e165529b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:00:06.522030 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a2c311-51ea-41b4-af13-8aa8e165529b None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:00:06.522885 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a2c311-51ea-41b4-af13-8aa8e165529b None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:00:06.523344 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a2c311-51ea-41b4-af13-8aa8e165529b None admin] PATH_INFO: `/v3/users` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:00:06.524670 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a2c311-51ea-41b4-af13-8aa8e165529b None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 22:00:06.525194 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a2c311-51ea-41b4-af13-8aa8e165529b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:00:06.528461 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07a2c311-51ea-41b4-af13-8aa8e165529b None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:00:06.553970 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 466/938] 10.222.0.52 () {56 vars in 1149 bytes} [Wed Jul 13 22:00:06 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 204 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 22:00:06.555034 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a2c311-51ea-41b4-af13-8aa8e165529b None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:00:06.595592 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06dbf5ae-9955-41b8-bbc0-0a3e2d6fb7c1 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:00:06.630149 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:00:06.645441 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b8f344-17ab-4026-b2b1-151e0ed51109 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:00:06.646344 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b8f344-17ab-4026-b2b1-151e0ed51109 None admin] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:00:06.646845 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b8f344-17ab-4026-b2b1-151e0ed51109 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:00:06.647275 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b8f344-17ab-4026-b2b1-151e0ed51109 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:00:06.648155 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b8f344-17ab-4026-b2b1-151e0ed51109 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:00:06.649758 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69b8f344-17ab-4026-b2b1-151e0ed51109 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:00:06.676631 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 473/939] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Jul 13 22:00:06 2022] POST /identity/v3/endpoints => generated 347 bytes in 222 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 22:00:06.682542 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a2c311-51ea-41b4-af13-8aa8e165529b None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:00:06.728863 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f972a5e-611c-4c7d-b991-202d159f7e53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:00:06.729973 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f972a5e-611c-4c7d-b991-202d159f7e53 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:00:06.730404 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f972a5e-611c-4c7d-b991-202d159f7e53 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:00:06.730850 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f972a5e-611c-4c7d-b991-202d159f7e53 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:00:06.743375 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f972a5e-611c-4c7d-b991-202d159f7e53 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:00:06.745208 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f972a5e-611c-4c7d-b991-202d159f7e53 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:00:06.765704 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application [None req-1f972a5e-611c-4c7d-b991-202d159f7e53 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 22:00:06.765704 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 22:00:06.765704 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 22:00:06.765704 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 22:00:06.765704 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 22:00:06.765704 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 22:00:06.765704 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 22:00:06.765704 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 22:00:06.765704 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 22:00:06.765704 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 22:00:06.765704 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 22:00:06.765704 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 22:00:06.765704 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 22:00:06.765704 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 22:00:06.765704 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 22:00:06.766919 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 22:00:06.766919 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 22:00:06.766919 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 22:00:06.766919 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 22:00:06.766919 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 22:00:06.766919 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 22:00:06.766919 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 22:00:06.766919 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 22:00:06.766919 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application with Lock( Jul 13 22:00:06.766919 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 22:00:06.766919 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._enter() Jul 13 22:00:06.766919 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 22:00:06.766919 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 22:00:06.766919 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 22:00:06.766919 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.creator() Jul 13 22:00:06.766919 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 22:00:06.766919 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application created_value = creator( Jul 13 22:00:06.768070 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 22:00:06.768070 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 22:00:06.768070 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 22:00:06.768070 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Jul 13 22:00:06.770165 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 467/940] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Jul 13 22:00:06 2022] GET /identity/v3/domains/Default => generated 87 bytes in 215 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 22:00:06.776602 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b8f344-17ab-4026-b2b1-151e0ed51109 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:00:06.819925 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ceff72-be98-4a37-b8dc-012fd0876ed7 None None] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:00:06.820283 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ceff72-be98-4a37-b8dc-012fd0876ed7 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:00:06.820472 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4ceff72-be98-4a37-b8dc-012fd0876ed7 None None] PATH_INFO: `/` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:00:06.822298 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 468/941] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jul 13 22:00:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 13 22:00:06.829301 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4881c43-b95e-44aa-a616-f545669da122 None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:00:06.829957 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4881c43-b95e-44aa-a616-f545669da122 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:00:06.830874 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4881c43-b95e-44aa-a616-f545669da122 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:00:06.853731 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-f4881c43-b95e-44aa-a616-f545669da122 None None] MFA Rules not processed for user `cc99c93da5074cf5a48bc2aa9a92c5e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 22:00:06.855424 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-f4881c43-b95e-44aa-a616-f545669da122 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:00:06.882130 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:00:06.906185 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69b8f344-17ab-4026-b2b1-151e0ed51109 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:00:06.946605 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 469/942] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jul 13 22:00:06 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 13 22:00:06.953859 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2394ea1-d6cf-4a11-b200-65c0881d469c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:00:06.954575 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2394ea1-d6cf-4a11-b200-65c0881d469c None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:00:06.954858 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2394ea1-d6cf-4a11-b200-65c0881d469c None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:00:06.955132 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2394ea1-d6cf-4a11-b200-65c0881d469c None admin] PATH_INFO: `/v3/domains` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:00:06.955861 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2394ea1-d6cf-4a11-b200-65c0881d469c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 22:00:06.956164 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2394ea1-d6cf-4a11-b200-65c0881d469c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:00:06.957547 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2394ea1-d6cf-4a11-b200-65c0881d469c None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:00:06.970691 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 474/943] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Jul 13 22:00:06 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 195 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 22:00:06.972547 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4881c43-b95e-44aa-a616-f545669da122 None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:00:06.975733 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2394ea1-d6cf-4a11-b200-65c0881d469c None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:00:07.029722 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:00:07.091461 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2394ea1-d6cf-4a11-b200-65c0881d469c None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:00:07.113986 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4881c43-b95e-44aa-a616-f545669da122 None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:00:07.129627 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31fb6b49-3241-47c8-888c-dbb5fce87001 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:00:07.130528 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fb6b49-3241-47c8-888c-dbb5fce87001 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:00:07.130876 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fb6b49-3241-47c8-888c-dbb5fce87001 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:00:07.131210 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31fb6b49-3241-47c8-888c-dbb5fce87001 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:00:07.138623 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31fb6b49-3241-47c8-888c-dbb5fce87001 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:00:07.140257 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31fb6b49-3241-47c8-888c-dbb5fce87001 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:00:07.155602 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application [None req-31fb6b49-3241-47c8-888c-dbb5fce87001 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 13 22:00:07.155602 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 22:00:07.155602 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 22:00:07.155602 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 22:00:07.155602 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 22:00:07.155602 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 22:00:07.155602 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 22:00:07.155602 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 22:00:07.155602 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 22:00:07.155602 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 22:00:07.155602 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 22:00:07.155602 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 22:00:07.155602 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 13 22:00:07.155602 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 13 22:00:07.155602 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 13 22:00:07.156779 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 13 22:00:07.156779 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 22:00:07.156779 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 22:00:07.156779 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 22:00:07.156779 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 22:00:07.156779 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 22:00:07.156779 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 22:00:07.156779 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 22:00:07.156779 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application with Lock( Jul 13 22:00:07.156779 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 22:00:07.156779 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._enter() Jul 13 22:00:07.156779 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 22:00:07.156779 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 22:00:07.156779 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 22:00:07.156779 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.creator() Jul 13 22:00:07.156779 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 22:00:07.156779 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application created_value = creator( Jul 13 22:00:07.157928 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 13 22:00:07.157928 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 13 22:00:07.157928 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 13 22:00:07.157928 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 13 22:00:07.157928 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 13 22:00:07.157928 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 13 22:00:07.157928 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 13 22:00:07.157928 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Jul 13 22:00:07.159124 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 475/944] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Jul 13 22:00:06 2022] GET /identity/v3/projects/service => generated 88 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 13 22:00:07.159681 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53d4ccd-fa1b-4c7b-a13e-b837db83b495 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:00:07.160407 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53d4ccd-fa1b-4c7b-a13e-b837db83b495 None admin] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:00:07.160676 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53d4ccd-fa1b-4c7b-a13e-b837db83b495 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:00:07.160951 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b53d4ccd-fa1b-4c7b-a13e-b837db83b495 None admin] PATH_INFO: `/v3/projects` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:00:07.161679 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b53d4ccd-fa1b-4c7b-a13e-b837db83b495 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:00:07.163162 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b53d4ccd-fa1b-4c7b-a13e-b837db83b495 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:00:07.164812 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31fb6b49-3241-47c8-888c-dbb5fce87001 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:00:07.165041 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: WARNING keystone.server.flask.common [None req-b53d4ccd-fa1b-4c7b-a13e-b837db83b495 None admin] Deprecated: Not specifying a domain during a create user, group or project call, and relying on falling back to the default domain, is deprecated as of Liberty. There is no plan to remove this compatibility, however, future API versions may remove this, so please specify the domain explicitly or use a domain-scoped token. Jul 13 22:00:07.168337 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:00:07.190732 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.sql.core [None req-b53d4ccd-fa1b-4c7b-a13e-b837db83b495 None admin] Conflict project: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 13 22:00:07.190732 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jul 13 22:00:07.190732 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [parameters: {'id': '0bbc1c82c0244fdb85b089ec048a95fb', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jul 13 22:00:07.190732 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80382) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 13 22:00:07.202317 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application [None req-b53d4ccd-fa1b-4c7b-a13e-b837db83b495 None admin] Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 0bbc1c82c0244fdb85b089ec048a95fb.: keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 0bbc1c82c0244fdb85b089ec048a95fb. Jul 13 22:00:07.202317 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 22:00:07.202317 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 13 22:00:07.202317 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 13 22:00:07.202317 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 13 22:00:07.202317 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 13 22:00:07.202317 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 13 22:00:07.202317 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application result = self._query(query) Jul 13 22:00:07.202317 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 13 22:00:07.202317 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application conn.query(q) Jul 13 22:00:07.202317 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 13 22:00:07.202317 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 13 22:00:07.202317 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 13 22:00:07.203324 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application result.read() Jul 13 22:00:07.203324 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 13 22:00:07.203324 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 13 22:00:07.203324 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 13 22:00:07.203324 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 13 22:00:07.203324 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 13 22:00:07.203324 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 13 22:00:07.203324 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 13 22:00:07.203324 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 13 22:00:07.203324 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 13 22:00:07.203324 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Jul 13 22:00:07.203324 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 13 22:00:07.203324 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Jul 13 22:00:07.203324 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 22:00:07.203324 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 13 22:00:07.203324 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 13 22:00:07.203324 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 241, in create_project Jul 13 22:00:07.203324 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return project_ref.to_dict() Jul 13 22:00:07.204268 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 13 22:00:07.204268 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application next(self.gen) Jul 13 22:00:07.204268 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 13 22:00:07.204268 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application yield resource Jul 13 22:00:07.204268 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 13 22:00:07.204268 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application next(self.gen) Jul 13 22:00:07.204268 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 13 22:00:07.204268 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application self.session.rollback() Jul 13 22:00:07.204268 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 13 22:00:07.204268 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application self.force_reraise() Jul 13 22:00:07.204268 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 13 22:00:07.204268 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application raise self.value Jul 13 22:00:07.204268 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 13 22:00:07.204268 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 13 22:00:07.204268 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 13 22:00:07.204268 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application session.commit() Jul 13 22:00:07.204268 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 13 22:00:07.205185 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 13 22:00:07.205185 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 13 22:00:07.205185 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application self._prepare_impl() Jul 13 22:00:07.205185 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 13 22:00:07.205185 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application self.session.flush() Jul 13 22:00:07.205185 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 13 22:00:07.205185 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application self._flush(objects) Jul 13 22:00:07.205185 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 13 22:00:07.205185 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 13 22:00:07.205185 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 13 22:00:07.205185 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application compat.raise_( Jul 13 22:00:07.205185 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 13 22:00:07.205185 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application raise exception Jul 13 22:00:07.205185 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 13 22:00:07.205185 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application flush_context.execute() Jul 13 22:00:07.205185 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 13 22:00:07.205185 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application rec.execute(self) Jul 13 22:00:07.206809 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 13 22:00:07.206809 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 13 22:00:07.206809 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 13 22:00:07.206809 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 13 22:00:07.206809 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1097, in _emit_insert_statements Jul 13 22:00:07.206809 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application c = connection._execute_20( Jul 13 22:00:07.206809 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 13 22:00:07.206809 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 13 22:00:07.206809 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 13 22:00:07.206809 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 13 22:00:07.206809 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 13 22:00:07.206809 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 13 22:00:07.206809 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 13 22:00:07.206809 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 13 22:00:07.206809 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 13 22:00:07.207610 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 13 22:00:07.207610 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 13 22:00:07.207610 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application raise exception Jul 13 22:00:07.207610 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 13 22:00:07.207610 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 13 22:00:07.207610 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 13 22:00:07.207610 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 13 22:00:07.207610 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 13 22:00:07.207610 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application result = self._query(query) Jul 13 22:00:07.207610 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 13 22:00:07.207610 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application conn.query(q) Jul 13 22:00:07.207610 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 13 22:00:07.207610 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 13 22:00:07.207610 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 13 22:00:07.207610 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application result.read() Jul 13 22:00:07.207610 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 13 22:00:07.207610 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 13 22:00:07.208478 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 13 22:00:07.208478 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 13 22:00:07.208478 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 13 22:00:07.208478 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 13 22:00:07.208478 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 13 22:00:07.208478 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 13 22:00:07.208478 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-service' for key 'project.ixu_project_name_domain_id'") Jul 13 22:00:07.208478 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application [SQL: INSERT INTO project (id, name, domain_id, description, enabled, extra, parent_id, is_domain) VALUES (%(id)s, %(name)s, %(domain_id)s, %(description)s, %(enabled)s, %(extra)s, %(parent_id)s, %(is_domain)s)] Jul 13 22:00:07.208478 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application [parameters: {'id': '0bbc1c82c0244fdb85b089ec048a95fb', 'name': 'service', 'domain_id': 'default', 'description': '', 'enabled': 1, 'extra': '{}', 'parent_id': 'default', 'is_domain': 0}] Jul 13 22:00:07.208478 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 13 22:00:07.208478 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Jul 13 22:00:07.208478 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 13 22:00:07.208478 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Jul 13 22:00:07.208478 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 22:00:07.208478 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 216, in create_project Jul 13 22:00:07.210347 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application ret = self.driver.create_project(project_id, project) Jul 13 22:00:07.210347 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 13 22:00:07.210347 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 13 22:00:07.210347 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - Duplicate entry found with name service at domain ID default. Jul 13 22:00:07.210347 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Jul 13 22:00:07.210347 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 13 22:00:07.210347 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Jul 13 22:00:07.210347 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 22:00:07.210347 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 22:00:07.210347 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 22:00:07.210347 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 22:00:07.210347 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 22:00:07.210347 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 22:00:07.210347 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 22:00:07.210347 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 22:00:07.210347 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 22:00:07.210347 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 22:00:07.213022 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 22:00:07.213022 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 176, in post Jul 13 22:00:07.213022 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application ref = PROVIDERS.resource_api.create_project( Jul 13 22:00:07.213022 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 22:00:07.213022 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 22:00:07.213022 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 218, in create_project Jul 13 22:00:07.213022 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application raise exception.Conflict( Jul 13 22:00:07.213022 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store project - it is not permitted to have two projects with either the same name or same id in the same domain: name is service, project id 0bbc1c82c0244fdb85b089ec048a95fb. Jul 13 22:00:07.213022 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Jul 13 22:00:07.213022 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 470/945] 10.222.0.52 () {60 vars in 1140 bytes} [Wed Jul 13 22:00:06 2022] POST /identity/v3/projects => generated 262 bytes in 232 msecs (HTTP/1.1 409) 5 headers in 183 bytes (2 switches on core 0) Jul 13 22:00:07.213022 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53d4ccd-fa1b-4c7b-a13e-b837db83b495 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:00:07.287004 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31fb6b49-3241-47c8-888c-dbb5fce87001 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:00:07.325568 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb2e3a0-a398-49f3-bba3-c3fdf9015825 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:00:07.326565 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb2e3a0-a398-49f3-bba3-c3fdf9015825 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:00:07.326947 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb2e3a0-a398-49f3-bba3-c3fdf9015825 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:00:07.327316 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceb2e3a0-a398-49f3-bba3-c3fdf9015825 None admin] PATH_INFO: `/v3/projects` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:00:07.328209 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb2e3a0-a398-49f3-bba3-c3fdf9015825 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 22:00:07.328619 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb2e3a0-a398-49f3-bba3-c3fdf9015825 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:00:07.330344 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceb2e3a0-a398-49f3-bba3-c3fdf9015825 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:00:07.339724 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:00:07.344238 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 476/946] 10.222.0.52 () {56 vars in 1148 bytes} [Wed Jul 13 22:00:07 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 22:00:07.347593 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b53d4ccd-fa1b-4c7b-a13e-b837db83b495 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:00:07.350060 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb2e3a0-a398-49f3-bba3-c3fdf9015825 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:00:07.393828 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f9f39e-1376-43b4-aef4-dc2e49dcdcfa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:00:07.395179 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f9f39e-1376-43b4-aef4-dc2e49dcdcfa None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:00:07.395507 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f9f39e-1376-43b4-aef4-dc2e49dcdcfa None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:00:07.395780 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7f9f39e-1376-43b4-aef4-dc2e49dcdcfa None admin] PATH_INFO: `/v3/projects/service` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:00:07.405944 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f9f39e-1376-43b4-aef4-dc2e49dcdcfa None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:00:07.408231 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7f9f39e-1376-43b4-aef4-dc2e49dcdcfa None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:00:07.417341 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:00:07.430244 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application [None req-a7f9f39e-1376-43b4-aef4-dc2e49dcdcfa None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 13 22:00:07.430244 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 22:00:07.430244 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 22:00:07.430244 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 22:00:07.430244 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 22:00:07.430244 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 22:00:07.430244 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 22:00:07.430244 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 22:00:07.430244 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 22:00:07.430244 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 22:00:07.430244 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 22:00:07.430244 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 22:00:07.430244 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 13 22:00:07.430244 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 13 22:00:07.430244 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 13 22:00:07.431344 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 13 22:00:07.431344 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 22:00:07.431344 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 22:00:07.431344 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 22:00:07.431344 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 22:00:07.431344 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 22:00:07.431344 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 22:00:07.431344 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 22:00:07.431344 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application with Lock( Jul 13 22:00:07.431344 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 22:00:07.431344 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._enter() Jul 13 22:00:07.431344 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 22:00:07.431344 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 22:00:07.431344 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 22:00:07.431344 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.creator() Jul 13 22:00:07.431344 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 22:00:07.431344 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application created_value = creator( Jul 13 22:00:07.432444 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 13 22:00:07.432444 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 13 22:00:07.432444 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 13 22:00:07.432444 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 13 22:00:07.432444 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 13 22:00:07.432444 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 13 22:00:07.432444 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 13 22:00:07.432444 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Jul 13 22:00:07.439828 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 471/947] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Jul 13 22:00:07 2022] GET /identity/v3/projects/service => generated 88 bytes in 225 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 22:00:07.441868 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f9f39e-1376-43b4-aef4-dc2e49dcdcfa None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:00:07.466638 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceb2e3a0-a398-49f3-bba3-c3fdf9015825 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:00:07.506031 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0ef4f1-a1e9-4a0e-b125-fdc7e85f81f0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:00:07.506968 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0ef4f1-a1e9-4a0e-b125-fdc7e85f81f0 None admin] REQUEST_METHOD: `PUT` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:00:07.507307 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0ef4f1-a1e9-4a0e-b125-fdc7e85f81f0 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:00:07.507526 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0ef4f1-a1e9-4a0e-b125-fdc7e85f81f0 None admin] PATH_INFO: `/v3/projects/417e3e2f1333481795a089bef2e24bea/users/b866bacdafae4db6a32714c07c56515f/roles/3987457f5fe64b568601d117c2c0ba9d` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:00:07.510177 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0ef4f1-a1e9-4a0e-b125-fdc7e85f81f0 None admin] RBAC: Authorizing `identity:create_grant(project_id=417e3e2f1333481795a089bef2e24bea, user_id=b866bacdafae4db6a32714c07c56515f, role_id=3987457f5fe64b568601d117c2c0ba9d)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:00:07.511539 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0ef4f1-a1e9-4a0e-b125-fdc7e85f81f0 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:00:07.534657 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 477/948] 10.222.0.52 () {58 vars in 1439 bytes} [Wed Jul 13 22:00:07 2022] PUT /identity/v3/projects/417e3e2f1333481795a089bef2e24bea/users/b866bacdafae4db6a32714c07c56515f/roles/3987457f5fe64b568601d117c2c0ba9d => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 22:00:07.591373 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7f9f39e-1376-43b4-aef4-dc2e49dcdcfa None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:00:07.632526 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-057ee0fb-e364-40b9-9bd8-4be77842b344 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:00:07.634172 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057ee0fb-e364-40b9-9bd8-4be77842b344 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:00:07.634705 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057ee0fb-e364-40b9-9bd8-4be77842b344 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:00:07.635196 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-057ee0fb-e364-40b9-9bd8-4be77842b344 None admin] PATH_INFO: `/v3/projects` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:00:07.636188 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057ee0fb-e364-40b9-9bd8-4be77842b344 None admin] RBAC: Adding query filter params (name=service) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 22:00:07.636693 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057ee0fb-e364-40b9-9bd8-4be77842b344 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:00:07.638379 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-057ee0fb-e364-40b9-9bd8-4be77842b344 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:00:07.666633 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 472/949] 10.222.0.52 () {56 vars in 1112 bytes} [Wed Jul 13 22:00:07 2022] GET /identity/v3/projects?name=service => generated 411 bytes in 221 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 22:00:08.585394 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833e0eab-3143-4f3b-b1ed-c1a285f75a3c None None] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:00:08.585907 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833e0eab-3143-4f3b-b1ed-c1a285f75a3c None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:00:08.585999 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-833e0eab-3143-4f3b-b1ed-c1a285f75a3c None None] PATH_INFO: `/` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:00:08.587917 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 478/950] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jul 13 22:00:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 22:00:08.593226 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a4d0b1-85e7-407e-b1c4-6ff181883f43 None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:00:08.593657 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a4d0b1-85e7-407e-b1c4-6ff181883f43 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:00:08.593783 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2a4d0b1-85e7-407e-b1c4-6ff181883f43 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:00:08.611041 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-c2a4d0b1-85e7-407e-b1c4-6ff181883f43 None None] MFA Rules not processed for user `cc99c93da5074cf5a48bc2aa9a92c5e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 22:00:08.612558 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-c2a4d0b1-85e7-407e-b1c4-6ff181883f43 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:00:08.623572 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:00:08.687119 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 473/951] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jul 13 22:00:08 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 13 22:00:08.709749 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833e0eab-3143-4f3b-b1ed-c1a285f75a3c None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:00:08.818879 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-833e0eab-3143-4f3b-b1ed-c1a285f75a3c None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:00:08.822383 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd7406b-6e92-4ecb-91d0-66cffc9253a1 None None] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:00:08.822383 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd7406b-6e92-4ecb-91d0-66cffc9253a1 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:00:08.822383 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bd7406b-6e92-4ecb-91d0-66cffc9253a1 None None] PATH_INFO: `/` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:00:08.825721 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 474/952] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jul 13 22:00:08 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 13 22:00:08.830478 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b729f35-8e41-4e76-b5c2-1466b30239f9 None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:00:08.830874 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b729f35-8e41-4e76-b5c2-1466b30239f9 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:00:08.831218 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b729f35-8e41-4e76-b5c2-1466b30239f9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:00:08.849198 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-9b729f35-8e41-4e76-b5c2-1466b30239f9 None None] MFA Rules not processed for user `cc99c93da5074cf5a48bc2aa9a92c5e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 22:00:08.850588 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-9b729f35-8e41-4e76-b5c2-1466b30239f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:00:08.850910 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:00:08.860406 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f0bb56-f440-40b2-b7be-0b5508394b7d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:00:08.861375 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f0bb56-f440-40b2-b7be-0b5508394b7d None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:00:08.861663 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f0bb56-f440-40b2-b7be-0b5508394b7d None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:00:08.862000 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8f0bb56-f440-40b2-b7be-0b5508394b7d None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:00:08.862749 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f0bb56-f440-40b2-b7be-0b5508394b7d None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:00:08.864225 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c8f0bb56-f440-40b2-b7be-0b5508394b7d None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:00:08.879022 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application [None req-c8f0bb56-f440-40b2-b7be-0b5508394b7d None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 13 22:00:08.879022 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 22:00:08.879022 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 13 22:00:08.879022 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 13 22:00:08.879022 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 13 22:00:08.879022 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 13 22:00:08.879022 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 13 22:00:08.879022 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 13 22:00:08.879022 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 13 22:00:08.879022 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Jul 13 22:00:08.879022 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 13 22:00:08.879022 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Jul 13 22:00:08.879022 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 22:00:08.879022 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 22:00:08.879022 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 22:00:08.879022 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 22:00:08.879022 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 22:00:08.880078 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 22:00:08.880078 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 22:00:08.880078 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 22:00:08.880078 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 22:00:08.880078 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 22:00:08.880078 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 22:00:08.880078 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 13 22:00:08.880078 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 13 22:00:08.880078 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 13 22:00:08.880078 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 13 22:00:08.880078 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 22:00:08.880078 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 22:00:08.880078 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 22:00:08.880078 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 22:00:08.880078 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 22:00:08.880078 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 22:00:08.881062 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 22:00:08.881062 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application with Lock( Jul 13 22:00:08.881062 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 22:00:08.881062 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._enter() Jul 13 22:00:08.881062 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 22:00:08.881062 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 22:00:08.881062 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 22:00:08.881062 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.creator() Jul 13 22:00:08.881062 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 22:00:08.881062 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application created_value = creator( Jul 13 22:00:08.881062 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 13 22:00:08.881062 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 13 22:00:08.881062 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 13 22:00:08.881062 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Jul 13 22:00:08.881062 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 479/953] 10.222.0.52 () {56 vars in 1126 bytes} [Wed Jul 13 22:00:08 2022] GET /identity/v3/services/object-store => generated 93 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 22:00:08.886688 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f0bb56-f440-40b2-b7be-0b5508394b7d None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:00:08.930631 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 475/954] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jul 13 22:00:08 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 13 22:00:08.950302 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b729f35-8e41-4e76-b5c2-1466b30239f9 None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:00:09.016879 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c8f0bb56-f440-40b2-b7be-0b5508394b7d None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:00:09.031533 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:00:09.059287 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b729f35-8e41-4e76-b5c2-1466b30239f9 None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:00:09.061430 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf417a9-ea0e-44bf-aac9-afdbe4cf22fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:00:09.062192 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf417a9-ea0e-44bf-aac9-afdbe4cf22fd None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:00:09.062571 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf417a9-ea0e-44bf-aac9-afdbe4cf22fd None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:00:09.062908 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cf417a9-ea0e-44bf-aac9-afdbe4cf22fd None admin] PATH_INFO: `/v3/services` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:00:09.063714 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf417a9-ea0e-44bf-aac9-afdbe4cf22fd None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 22:00:09.064081 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf417a9-ea0e-44bf-aac9-afdbe4cf22fd None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:00:09.065625 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cf417a9-ea0e-44bf-aac9-afdbe4cf22fd None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:00:09.071629 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 480/955] 10.222.0.52 () {56 vars in 1122 bytes} [Wed Jul 13 22:00:08 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 22:00:09.077094 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf417a9-ea0e-44bf-aac9-afdbe4cf22fd None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:00:09.105225 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0a35da-5439-4af8-adeb-61d5f1ddf7ae None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:00:09.106042 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0a35da-5439-4af8-adeb-61d5f1ddf7ae None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:00:09.106320 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0a35da-5439-4af8-adeb-61d5f1ddf7ae None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:00:09.106624 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b0a35da-5439-4af8-adeb-61d5f1ddf7ae None admin] PATH_INFO: `/v3/services/compute` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:00:09.107841 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0a35da-5439-4af8-adeb-61d5f1ddf7ae None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:00:09.109630 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b0a35da-5439-4af8-adeb-61d5f1ddf7ae None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:00:09.125582 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application [None req-9b0a35da-5439-4af8-adeb-61d5f1ddf7ae None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 13 22:00:09.125582 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 22:00:09.125582 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 13 22:00:09.125582 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 13 22:00:09.125582 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 13 22:00:09.125582 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 13 22:00:09.125582 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 13 22:00:09.125582 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 13 22:00:09.125582 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 13 22:00:09.125582 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Jul 13 22:00:09.125582 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 13 22:00:09.125582 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Jul 13 22:00:09.125582 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 22:00:09.125582 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 22:00:09.125582 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 22:00:09.125582 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 22:00:09.125582 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 22:00:09.127600 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 22:00:09.127600 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 22:00:09.127600 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 22:00:09.127600 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 22:00:09.127600 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 22:00:09.127600 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 22:00:09.127600 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 13 22:00:09.127600 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 13 22:00:09.127600 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 13 22:00:09.127600 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 13 22:00:09.127600 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 22:00:09.127600 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 22:00:09.127600 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 22:00:09.127600 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 22:00:09.127600 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 22:00:09.127600 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 22:00:09.128854 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 22:00:09.128854 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application with Lock( Jul 13 22:00:09.128854 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 22:00:09.128854 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._enter() Jul 13 22:00:09.128854 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 22:00:09.128854 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 22:00:09.128854 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 22:00:09.128854 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.creator() Jul 13 22:00:09.128854 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 22:00:09.128854 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application created_value = creator( Jul 13 22:00:09.128854 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 13 22:00:09.128854 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 13 22:00:09.128854 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 13 22:00:09.128854 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Jul 13 22:00:09.128854 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 476/956] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Jul 13 22:00:08 2022] GET /identity/v3/services/compute => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 13 22:00:09.136550 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0a35da-5439-4af8-adeb-61d5f1ddf7ae None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:00:09.143659 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:00:09.209445 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cf417a9-ea0e-44bf-aac9-afdbe4cf22fd None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:00:09.251707 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be90c6d8-7a10-4f84-a6a0-21923acaab59 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:00:09.252423 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be90c6d8-7a10-4f84-a6a0-21923acaab59 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:00:09.252700 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be90c6d8-7a10-4f84-a6a0-21923acaab59 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:00:09.253371 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be90c6d8-7a10-4f84-a6a0-21923acaab59 None admin] PATH_INFO: `/v3/services` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:00:09.254219 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be90c6d8-7a10-4f84-a6a0-21923acaab59 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 22:00:09.254542 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be90c6d8-7a10-4f84-a6a0-21923acaab59 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:00:09.256225 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be90c6d8-7a10-4f84-a6a0-21923acaab59 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:00:09.267701 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 481/957] 10.222.0.52 () {56 vars in 1122 bytes} [Wed Jul 13 22:00:09 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 187 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 22:00:09.269286 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be90c6d8-7a10-4f84-a6a0-21923acaab59 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:00:09.272245 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b0a35da-5439-4af8-adeb-61d5f1ddf7ae None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:00:09.309265 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28bdd239-4dc1-4a19-a77b-dfdf00ecfc98 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:00:09.309971 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bdd239-4dc1-4a19-a77b-dfdf00ecfc98 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:00:09.310271 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bdd239-4dc1-4a19-a77b-dfdf00ecfc98 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:00:09.310578 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28bdd239-4dc1-4a19-a77b-dfdf00ecfc98 None admin] PATH_INFO: `/v3/services` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:00:09.311275 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28bdd239-4dc1-4a19-a77b-dfdf00ecfc98 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 22:00:09.311574 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28bdd239-4dc1-4a19-a77b-dfdf00ecfc98 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:00:09.313119 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28bdd239-4dc1-4a19-a77b-dfdf00ecfc98 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:00:09.323151 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 477/958] 10.222.0.52 () {56 vars in 1112 bytes} [Wed Jul 13 22:00:09 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 22:00:09.324789 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28bdd239-4dc1-4a19-a77b-dfdf00ecfc98 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:00:09.329528 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:00:09.404495 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be90c6d8-7a10-4f84-a6a0-21923acaab59 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:00:09.424706 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:00:09.436613 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28bdd239-4dc1-4a19-a77b-dfdf00ecfc98 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:00:09.447008 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d5366bc0-1b49-45ef-842d-e3e166e78c18 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:00:09.447679 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5366bc0-1b49-45ef-842d-e3e166e78c18 None admin] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:00:09.447958 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5366bc0-1b49-45ef-842d-e3e166e78c18 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:00:09.448229 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d5366bc0-1b49-45ef-842d-e3e166e78c18 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:00:09.449323 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5366bc0-1b49-45ef-842d-e3e166e78c18 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:00:09.450723 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d5366bc0-1b49-45ef-842d-e3e166e78c18 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:00:09.482517 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 482/959] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Jul 13 22:00:09 2022] POST /identity/v3/endpoints => generated 349 bytes in 205 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 22:00:09.496043 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df4fef27-2b5e-4f0a-a8c2-3420042f2d58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:00:09.496725 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4fef27-2b5e-4f0a-a8c2-3420042f2d58 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:00:09.497000 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4fef27-2b5e-4f0a-a8c2-3420042f2d58 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:00:09.497269 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df4fef27-2b5e-4f0a-a8c2-3420042f2d58 None admin] PATH_INFO: `/v3/services` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:00:09.497990 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4fef27-2b5e-4f0a-a8c2-3420042f2d58 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 22:00:09.498294 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4fef27-2b5e-4f0a-a8c2-3420042f2d58 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:00:09.499875 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-df4fef27-2b5e-4f0a-a8c2-3420042f2d58 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:00:09.509681 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 478/960] 10.222.0.52 () {56 vars in 1112 bytes} [Wed Jul 13 22:00:09 2022] GET /identity/v3/services?type=compute => generated 124 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 22:00:09.621779 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a654e1-7b7b-40f0-8560-b48b7131bf5b None None] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:00:09.622321 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a654e1-7b7b-40f0-8560-b48b7131bf5b None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:00:09.622321 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0a654e1-7b7b-40f0-8560-b48b7131bf5b None None] PATH_INFO: `/` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:00:09.624117 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 483/961] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jul 13 22:00:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 22:00:09.630219 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187fd976-f005-443c-9f7f-db5b13b70a1c None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:00:09.630786 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187fd976-f005-443c-9f7f-db5b13b70a1c None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:00:09.630909 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-187fd976-f005-443c-9f7f-db5b13b70a1c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:00:09.654778 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-187fd976-f005-443c-9f7f-db5b13b70a1c None None] MFA Rules not processed for user `cc99c93da5074cf5a48bc2aa9a92c5e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 22:00:09.656012 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-187fd976-f005-443c-9f7f-db5b13b70a1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:00:09.725199 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:00:09.738539 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 479/962] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jul 13 22:00:09 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 13 22:00:09.761404 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a654e1-7b7b-40f0-8560-b48b7131bf5b None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:00:09.778845 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79712839-23d3-434a-8697-5b05bb70a6ea None None] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:00:09.779154 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79712839-23d3-434a-8697-5b05bb70a6ea None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:00:09.779397 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79712839-23d3-434a-8697-5b05bb70a6ea None None] PATH_INFO: `/` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:00:09.781033 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 480/963] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jul 13 22:00:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 13 22:00:09.785782 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a1bcab-9e95-402b-8188-99e864939a25 None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:00:09.786055 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a1bcab-9e95-402b-8188-99e864939a25 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:00:09.786331 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1a1bcab-9e95-402b-8188-99e864939a25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:00:09.803831 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-b1a1bcab-9e95-402b-8188-99e864939a25 None None] MFA Rules not processed for user `cc99c93da5074cf5a48bc2aa9a92c5e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 22:00:09.805044 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-b1a1bcab-9e95-402b-8188-99e864939a25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:00:09.877378 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 481/964] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jul 13 22:00:09 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 13 22:00:09.888263 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0a654e1-7b7b-40f0-8560-b48b7131bf5b None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:00:09.902925 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a1bcab-9e95-402b-8188-99e864939a25 None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:00:09.934180 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3bfd8d7-28c6-4679-b108-72b93d734dcf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:00:09.935721 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bfd8d7-28c6-4679-b108-72b93d734dcf None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:00:09.936014 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bfd8d7-28c6-4679-b108-72b93d734dcf None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:00:09.936292 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3bfd8d7-28c6-4679-b108-72b93d734dcf None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:00:09.950532 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3bfd8d7-28c6-4679-b108-72b93d734dcf None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 22:00:09.960126 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application [None req-b3bfd8d7-28c6-4679-b108-72b93d734dcf None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 13 22:00:09.960126 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 22:00:09.960126 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 22:00:09.960126 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 22:00:09.960126 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 22:00:09.960126 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 22:00:09.960126 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 22:00:09.960126 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 22:00:09.960126 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 22:00:09.960126 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 22:00:09.960126 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 22:00:09.960126 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 22:00:09.960126 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 22:00:09.960126 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 22:00:09.960126 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 22:00:09.960126 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 22:00:09.961141 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 22:00:09.961141 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 22:00:09.961141 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 22:00:09.961141 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 22:00:09.961141 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 22:00:09.961141 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 22:00:09.961141 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 22:00:09.961141 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 22:00:09.961141 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 22:00:09.961141 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 22:00:09.961141 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 22:00:09.961141 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application with Lock( Jul 13 22:00:09.961141 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 22:00:09.961141 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._enter() Jul 13 22:00:09.961141 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 22:00:09.961141 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 22:00:09.962021 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 22:00:09.962021 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.creator() Jul 13 22:00:09.962021 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 22:00:09.962021 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application created_value = creator( Jul 13 22:00:09.962021 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 22:00:09.962021 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 22:00:09.962021 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 22:00:09.962021 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 22:00:09.962021 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 22:00:09.962021 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 22:00:09.962021 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 13 22:00:09.962021 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Jul 13 22:00:09.967178 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 484/965] 10.222.0.52 () {56 vars in 1120 bytes} [Wed Jul 13 22:00:09 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 201 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 22:00:09.968144 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3bfd8d7-28c6-4679-b108-72b93d734dcf None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:00:10.000325 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:00:10.023950 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1a1bcab-9e95-402b-8188-99e864939a25 None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:00:10.054324 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:00:10.069189 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd02209d-0313-4cbf-8c3f-8edf9431d53e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:00:10.070717 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd02209d-0313-4cbf-8c3f-8edf9431d53e None admin] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:00:10.071180 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd02209d-0313-4cbf-8c3f-8edf9431d53e None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:00:10.071597 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd02209d-0313-4cbf-8c3f-8edf9431d53e None admin] PATH_INFO: `/v3/users` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:00:10.072631 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd02209d-0313-4cbf-8c3f-8edf9431d53e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:00:10.075147 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd02209d-0313-4cbf-8c3f-8edf9431d53e None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:00:10.078977 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3bfd8d7-28c6-4679-b108-72b93d734dcf None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:00:10.092523 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.sql.core [None req-bd02209d-0313-4cbf-8c3f-8edf9431d53e None admin] Conflict user: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 13 22:00:10.092523 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jul 13 22:00:10.092523 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [parameters: {'user_id': 'cbd44d520fdf4eaa8ba31e9f4ddfddb3', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jul 13 22:00:10.092523 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: (Background on this error at: https://sqlalche.me/e/14/gkpj) {{(pid=80382) wrapper /opt/stack/keystone/keystone/common/sql/core.py:533}} Jul 13 22:00:10.103123 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application [None req-bd02209d-0313-4cbf-8c3f-8edf9431d53e None admin] Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default.: keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jul 13 22:00:10.103123 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 22:00:10.103123 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 13 22:00:10.103123 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 13 22:00:10.103123 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 13 22:00:10.103123 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 13 22:00:10.103123 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 13 22:00:10.103123 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application result = self._query(query) Jul 13 22:00:10.103123 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 13 22:00:10.103123 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application conn.query(q) Jul 13 22:00:10.103123 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 13 22:00:10.103123 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 13 22:00:10.103123 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 13 22:00:10.103123 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application result.read() Jul 13 22:00:10.103123 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 13 22:00:10.104294 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 13 22:00:10.104294 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 13 22:00:10.104294 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 13 22:00:10.104294 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 13 22:00:10.104294 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 13 22:00:10.104294 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 13 22:00:10.104294 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 13 22:00:10.104294 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application pymysql.err.IntegrityError: (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 13 22:00:10.104294 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Jul 13 22:00:10.104294 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application The above exception was the direct cause of the following exception: Jul 13 22:00:10.104294 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Jul 13 22:00:10.104294 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 22:00:10.104294 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 528, in wrapper Jul 13 22:00:10.104294 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return method(*args, **kwargs) Jul 13 22:00:10.104294 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 141, in create_user Jul 13 22:00:10.104294 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return base.filter_user(user_ref.to_dict()) Jul 13 22:00:10.104294 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 13 22:00:10.104294 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application next(self.gen) Jul 13 22:00:10.105475 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 1060, in _transaction_scope Jul 13 22:00:10.105475 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application yield resource Jul 13 22:00:10.105475 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/lib/python3.8/contextlib.py", line 120, in __exit__ Jul 13 22:00:10.105475 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application next(self.gen) Jul 13 22:00:10.105475 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 662, in _session Jul 13 22:00:10.105475 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application self.session.rollback() Jul 13 22:00:10.105475 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 227, in __exit__ Jul 13 22:00:10.105475 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application self.force_reraise() Jul 13 22:00:10.105475 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_utils/excutils.py", line 200, in force_reraise Jul 13 22:00:10.105475 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application raise self.value Jul 13 22:00:10.105475 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 659, in _session Jul 13 22:00:10.105475 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application self._end_session_transaction(self.session) Jul 13 22:00:10.105475 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/oslo_db/sqlalchemy/enginefacade.py", line 687, in _end_session_transaction Jul 13 22:00:10.105475 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application session.commit() Jul 13 22:00:10.105475 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 1451, in commit Jul 13 22:00:10.105475 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application self._transaction.commit(_to_root=self.future) Jul 13 22:00:10.105475 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 829, in commit Jul 13 22:00:10.106610 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application self._prepare_impl() Jul 13 22:00:10.106610 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 808, in _prepare_impl Jul 13 22:00:10.106610 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application self.session.flush() Jul 13 22:00:10.106610 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3383, in flush Jul 13 22:00:10.106610 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application self._flush(objects) Jul 13 22:00:10.106610 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3523, in _flush Jul 13 22:00:10.106610 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application transaction.rollback(_capture_exception=True) Jul 13 22:00:10.106610 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/langhelpers.py", line 70, in __exit__ Jul 13 22:00:10.106610 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application compat.raise_( Jul 13 22:00:10.106610 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 13 22:00:10.106610 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application raise exception Jul 13 22:00:10.106610 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/session.py", line 3483, in _flush Jul 13 22:00:10.106610 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application flush_context.execute() Jul 13 22:00:10.106610 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 456, in execute Jul 13 22:00:10.106610 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application rec.execute(self) Jul 13 22:00:10.106610 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/unitofwork.py", line 630, in execute Jul 13 22:00:10.106610 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application util.preloaded.orm_persistence.save_obj( Jul 13 22:00:10.107728 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 245, in save_obj Jul 13 22:00:10.107728 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application _emit_insert_statements( Jul 13 22:00:10.107728 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/orm/persistence.py", line 1238, in _emit_insert_statements Jul 13 22:00:10.107728 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application result = connection._execute_20( Jul 13 22:00:10.107728 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1631, in _execute_20 Jul 13 22:00:10.107728 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return meth(self, args_10style, kwargs_10style, execution_options) Jul 13 22:00:10.107728 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/sql/elements.py", line 332, in _execute_on_connection Jul 13 22:00:10.107728 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return connection._execute_clauseelement( Jul 13 22:00:10.107728 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1498, in _execute_clauseelement Jul 13 22:00:10.107728 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application ret = self._execute_context( Jul 13 22:00:10.107728 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1862, in _execute_context Jul 13 22:00:10.107728 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application self._handle_dbapi_exception( Jul 13 22:00:10.107728 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 2041, in _handle_dbapi_exception Jul 13 22:00:10.107728 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application util.raise_(newraise, with_traceback=exc_info[2], from_=e) Jul 13 22:00:10.107728 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/util/compat.py", line 208, in raise_ Jul 13 22:00:10.107728 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application raise exception Jul 13 22:00:10.108783 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/base.py", line 1819, in _execute_context Jul 13 22:00:10.108783 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application self.dialect.do_execute( Jul 13 22:00:10.108783 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/sqlalchemy/engine/default.py", line 732, in do_execute Jul 13 22:00:10.108783 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application cursor.execute(statement, parameters) Jul 13 22:00:10.108783 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 148, in execute Jul 13 22:00:10.108783 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application result = self._query(query) Jul 13 22:00:10.108783 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/cursors.py", line 310, in _query Jul 13 22:00:10.108783 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application conn.query(q) Jul 13 22:00:10.108783 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 548, in query Jul 13 22:00:10.108783 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application self._affected_rows = self._read_query_result(unbuffered=unbuffered) Jul 13 22:00:10.108783 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 775, in _read_query_result Jul 13 22:00:10.108783 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application result.read() Jul 13 22:00:10.108783 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 1156, in read Jul 13 22:00:10.108783 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application first_packet = self.connection._read_packet() Jul 13 22:00:10.108783 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/connections.py", line 725, in _read_packet Jul 13 22:00:10.108783 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application packet.raise_for_error() Jul 13 22:00:10.109829 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/protocol.py", line 221, in raise_for_error Jul 13 22:00:10.109829 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application err.raise_mysql_exception(self._data) Jul 13 22:00:10.109829 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/pymysql/err.py", line 143, in raise_mysql_exception Jul 13 22:00:10.109829 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application raise errorclass(errno, errval) Jul 13 22:00:10.109829 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application oslo_db.exception.DBDuplicateEntry: (pymysql.err.IntegrityError) (1062, "Duplicate entry 'default-cinder' for key 'local_user.domain_id'") Jul 13 22:00:10.109829 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application [SQL: INSERT INTO local_user (user_id, domain_id, name, failed_auth_count, failed_auth_at) VALUES (%(user_id)s, %(domain_id)s, %(name)s, %(failed_auth_count)s, %(failed_auth_at)s)] Jul 13 22:00:10.109829 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application [parameters: {'user_id': 'cbd44d520fdf4eaa8ba31e9f4ddfddb3', 'domain_id': 'default', 'name': 'cinder', 'failed_auth_count': 0, 'failed_auth_at': None}] Jul 13 22:00:10.109829 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application (Background on this error at: https://sqlalche.me/e/14/gkpj) Jul 13 22:00:10.109829 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Jul 13 22:00:10.109829 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 13 22:00:10.109829 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Jul 13 22:00:10.109829 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 22:00:10.109829 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 22:00:10.109829 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 22:00:10.109829 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 22:00:10.111283 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 22:00:10.111283 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 22:00:10.111283 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 22:00:10.111283 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 22:00:10.111283 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 22:00:10.111283 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 22:00:10.111283 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 22:00:10.111283 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 244, in post Jul 13 22:00:10.111283 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.create_user( Jul 13 22:00:10.111283 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 22:00:10.111283 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 22:00:10.111283 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 13 22:00:10.111283 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 22:00:10.111283 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 13 22:00:10.111283 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 22:00:10.111283 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1000, in create_user Jul 13 22:00:10.111283 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application ref = self._create_user_with_federated_objects(user, driver) Jul 13 22:00:10.113463 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 966, in _create_user_with_federated_objects Jul 13 22:00:10.113463 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application user = driver.create_user(user['id'], user) Jul 13 22:00:10.113463 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/sql/core.py", line 563, in wrapper Jul 13 22:00:10.113463 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application raise exception.Conflict(type=conflict_type, Jul 13 22:00:10.113463 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application keystone.exception.Conflict: Conflict occurred attempting to store user - Duplicate entry found with name cinder at domain ID default. Jul 13 22:00:10.113463 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Jul 13 22:00:10.114901 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 482/966] 10.222.0.52 () {60 vars in 1131 bytes} [Wed Jul 13 22:00:09 2022] POST /identity/v3/users => generated 160 bytes in 212 msecs (HTTP/1.1 409) 5 headers in 183 bytes (1 switches on core 0) Jul 13 22:00:10.118294 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd02209d-0313-4cbf-8c3f-8edf9431d53e None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:00:10.130652 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9efb1760-10bd-406d-b9e4-8e3486bbef97 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:00:10.131346 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efb1760-10bd-406d-b9e4-8e3486bbef97 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:00:10.131718 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efb1760-10bd-406d-b9e4-8e3486bbef97 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:00:10.132079 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9efb1760-10bd-406d-b9e4-8e3486bbef97 None admin] PATH_INFO: `/v3/roles` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:00:10.132934 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9efb1760-10bd-406d-b9e4-8e3486bbef97 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 22:00:10.133364 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9efb1760-10bd-406d-b9e4-8e3486bbef97 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:00:10.135234 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9efb1760-10bd-406d-b9e4-8e3486bbef97 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:00:10.148488 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 485/967] 10.222.0.52 () {56 vars in 1115 bytes} [Wed Jul 13 22:00:09 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 22:00:10.149613 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9efb1760-10bd-406d-b9e4-8e3486bbef97 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:00:10.250797 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:00:10.259696 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd02209d-0313-4cbf-8c3f-8edf9431d53e None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:00:10.261593 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9efb1760-10bd-406d-b9e4-8e3486bbef97 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:00:10.299690 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d648409c-6ad5-4deb-b7ab-83cb15e69c58 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:00:10.300669 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d648409c-6ad5-4deb-b7ab-83cb15e69c58 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:00:10.301044 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d648409c-6ad5-4deb-b7ab-83cb15e69c58 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:00:10.301414 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d648409c-6ad5-4deb-b7ab-83cb15e69c58 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:00:10.302930 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc176ca-fb31-48c4-83a1-eb93e9aeff6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:00:10.304104 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc176ca-fb31-48c4-83a1-eb93e9aeff6b None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:00:10.304856 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc176ca-fb31-48c4-83a1-eb93e9aeff6b None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:00:10.305512 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fc176ca-fb31-48c4-83a1-eb93e9aeff6b None admin] PATH_INFO: `/v3/users/cinder` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:00:10.313750 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d648409c-6ad5-4deb-b7ab-83cb15e69c58 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:00:10.315354 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d648409c-6ad5-4deb-b7ab-83cb15e69c58 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:00:10.316146 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc176ca-fb31-48c4-83a1-eb93e9aeff6b None admin] RBAC: Authorizing `identity:get_user(user_id=cinder)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:00:10.317946 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fc176ca-fb31-48c4-83a1-eb93e9aeff6b None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:00:10.334385 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application [None req-d648409c-6ad5-4deb-b7ab-83cb15e69c58 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 22:00:10.334385 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 22:00:10.334385 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 22:00:10.334385 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 22:00:10.334385 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 22:00:10.334385 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 22:00:10.334385 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 22:00:10.334385 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 22:00:10.334385 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 22:00:10.334385 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 22:00:10.334385 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 22:00:10.334385 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 22:00:10.334385 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 22:00:10.334385 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 22:00:10.334385 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 22:00:10.335319 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 22:00:10.335319 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 22:00:10.335319 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 22:00:10.335319 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 22:00:10.335319 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 22:00:10.335319 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 22:00:10.335319 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 22:00:10.335319 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 22:00:10.335319 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application with Lock( Jul 13 22:00:10.335319 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 22:00:10.335319 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._enter() Jul 13 22:00:10.335319 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 22:00:10.335319 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 22:00:10.335319 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 22:00:10.335319 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.creator() Jul 13 22:00:10.335319 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 22:00:10.335319 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application created_value = creator( Jul 13 22:00:10.336530 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 22:00:10.336530 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 22:00:10.336530 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 22:00:10.336530 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Jul 13 22:00:10.337004 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application [None req-3fc176ca-fb31-48c4-83a1-eb93e9aeff6b None admin] Could not find user: cinder.: keystone.exception.UserNotFound: Could not find user: cinder. Jul 13 22:00:10.337004 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 22:00:10.337004 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 22:00:10.337004 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 22:00:10.337004 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 22:00:10.337004 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 22:00:10.337004 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 22:00:10.337004 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 22:00:10.337004 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 22:00:10.337004 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 22:00:10.337004 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 22:00:10.337004 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 22:00:10.337004 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 13 22:00:10.337004 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 13 22:00:10.337004 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 13 22:00:10.337004 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 13 22:00:10.337893 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 22:00:10.337893 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 22:00:10.337893 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 13 22:00:10.337893 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 22:00:10.337893 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 13 22:00:10.337893 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 22:00:10.337893 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 22:00:10.337893 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 22:00:10.337893 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 22:00:10.337893 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 22:00:10.337893 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 22:00:10.337893 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application with Lock( Jul 13 22:00:10.337893 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 22:00:10.337893 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._enter() Jul 13 22:00:10.337893 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 22:00:10.337893 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 22:00:10.337893 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 22:00:10.337893 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.creator() Jul 13 22:00:10.338886 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 22:00:10.338886 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application created_value = creator( Jul 13 22:00:10.338886 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 13 22:00:10.338886 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 13 22:00:10.338886 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 13 22:00:10.338886 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 13 22:00:10.338886 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 13 22:00:10.338886 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 13 22:00:10.338886 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: cinder. Jul 13 22:00:10.338886 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Jul 13 22:00:10.340746 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 483/968] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Jul 13 22:00:10 2022] GET /identity/v3/users/cinder => generated 84 bytes in 220 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 13 22:00:10.342528 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc176ca-fb31-48c4-83a1-eb93e9aeff6b None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:00:10.346532 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 486/969] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Jul 13 22:00:10 2022] GET /identity/v3/domains/Default => generated 87 bytes in 193 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 22:00:10.347546 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d648409c-6ad5-4deb-b7ab-83cb15e69c58 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:00:10.398869 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:00:10.452385 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d648409c-6ad5-4deb-b7ab-83cb15e69c58 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:00:10.466188 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fc176ca-fb31-48c4-83a1-eb93e9aeff6b None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:00:10.488699 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f42578-4f2f-4489-9ee7-ee265d76fb80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:00:10.489531 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f42578-4f2f-4489-9ee7-ee265d76fb80 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:00:10.489953 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f42578-4f2f-4489-9ee7-ee265d76fb80 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:00:10.490319 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2f42578-4f2f-4489-9ee7-ee265d76fb80 None admin] PATH_INFO: `/v3/domains` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:00:10.491267 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f42578-4f2f-4489-9ee7-ee265d76fb80 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 22:00:10.491668 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f42578-4f2f-4489-9ee7-ee265d76fb80 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:00:10.493265 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2f42578-4f2f-4489-9ee7-ee265d76fb80 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:00:10.506822 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5cb4d7a-6402-46ab-a62c-5d6515dd05be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:00:10.507762 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cb4d7a-6402-46ab-a62c-5d6515dd05be None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:00:10.508363 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cb4d7a-6402-46ab-a62c-5d6515dd05be None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:00:10.508870 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5cb4d7a-6402-46ab-a62c-5d6515dd05be None admin] PATH_INFO: `/v3/users` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:00:10.509832 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5cb4d7a-6402-46ab-a62c-5d6515dd05be None admin] RBAC: Adding query filter params (name=cinder) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 22:00:10.510262 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5cb4d7a-6402-46ab-a62c-5d6515dd05be None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:00:10.511013 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 487/970] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Jul 13 22:00:10 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 22:00:10.512194 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5cb4d7a-6402-46ab-a62c-5d6515dd05be None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:00:10.512569 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f42578-4f2f-4489-9ee7-ee265d76fb80 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:00:10.516604 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:00:10.566449 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 484/971] 10.222.0.52 () {56 vars in 1101 bytes} [Wed Jul 13 22:00:10 2022] GET /identity/v3/users?name=cinder => generated 354 bytes in 219 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 22:00:10.634013 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2f42578-4f2f-4489-9ee7-ee265d76fb80 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:00:10.676476 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c0ab60-5e1c-4987-b94e-17d300d731fe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:00:10.677363 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c0ab60-5e1c-4987-b94e-17d300d731fe None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:00:10.677646 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c0ab60-5e1c-4987-b94e-17d300d731fe None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:00:10.677918 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-97c0ab60-5e1c-4987-b94e-17d300d731fe None admin] PATH_INFO: `/v3/users/glance-swift` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:00:10.687001 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c0ab60-5e1c-4987-b94e-17d300d731fe None admin] RBAC: Authorizing `identity:get_user(user_id=glance-swift)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:00:10.688495 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-97c0ab60-5e1c-4987-b94e-17d300d731fe None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:00:10.705367 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application [None req-97c0ab60-5e1c-4987-b94e-17d300d731fe None admin] Could not find user: glance-swift.: keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 13 22:00:10.705367 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 22:00:10.705367 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 22:00:10.705367 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 22:00:10.705367 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 22:00:10.705367 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 22:00:10.705367 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 22:00:10.705367 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 22:00:10.705367 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 22:00:10.705367 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 22:00:10.705367 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 22:00:10.705367 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 22:00:10.705367 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 13 22:00:10.705367 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 13 22:00:10.705367 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 13 22:00:10.705367 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 13 22:00:10.706413 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 22:00:10.706413 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 22:00:10.706413 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 13 22:00:10.706413 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 22:00:10.706413 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 13 22:00:10.706413 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 22:00:10.706413 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 22:00:10.706413 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 22:00:10.706413 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 22:00:10.706413 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 22:00:10.706413 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 22:00:10.706413 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application with Lock( Jul 13 22:00:10.706413 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 22:00:10.706413 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._enter() Jul 13 22:00:10.706413 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 22:00:10.706413 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 22:00:10.706413 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 22:00:10.706413 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.creator() Jul 13 22:00:10.708184 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 22:00:10.708184 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application created_value = creator( Jul 13 22:00:10.708184 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 13 22:00:10.708184 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 13 22:00:10.708184 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 13 22:00:10.708184 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 13 22:00:10.708184 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 13 22:00:10.708184 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 13 22:00:10.708184 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: glance-swift. Jul 13 22:00:10.708184 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Jul 13 22:00:10.708184 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 488/972] 10.222.0.52 () {56 vars in 1117 bytes} [Wed Jul 13 22:00:10 2022] GET /identity/v3/users/glance-swift => generated 90 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 22:00:10.712941 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5cb4d7a-6402-46ab-a62c-5d6515dd05be None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:00:10.819068 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5cb4d7a-6402-46ab-a62c-5d6515dd05be None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:00:10.821846 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:00:10.855731 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b859fec5-723a-46a7-a872-536c1ab2726b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:00:10.856414 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b859fec5-723a-46a7-a872-536c1ab2726b None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:00:10.856684 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b859fec5-723a-46a7-a872-536c1ab2726b None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:00:10.856961 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b859fec5-723a-46a7-a872-536c1ab2726b None admin] PATH_INFO: `/v3/users` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:00:10.857797 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b859fec5-723a-46a7-a872-536c1ab2726b None admin] RBAC: Adding query filter params (domain_id=default, name=glance-swift) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 22:00:10.858099 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b859fec5-723a-46a7-a872-536c1ab2726b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:00:10.859566 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b859fec5-723a-46a7-a872-536c1ab2726b None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:00:10.878433 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 485/973] 10.222.0.52 () {56 vars in 1149 bytes} [Wed Jul 13 22:00:10 2022] GET /identity/v3/users?domain_id=default&name=glance-swift => generated 384 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 22:00:10.884202 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c0ab60-5e1c-4987-b94e-17d300d731fe None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:00:10.952822 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:00:10.991071 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-97c0ab60-5e1c-4987-b94e-17d300d731fe None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:00:11.026809 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d954a90-57b1-4d25-b734-f5ed93dfee3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:00:11.027469 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d954a90-57b1-4d25-b734-f5ed93dfee3b None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:00:11.027748 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d954a90-57b1-4d25-b734-f5ed93dfee3b None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:00:11.028017 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d954a90-57b1-4d25-b734-f5ed93dfee3b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:00:11.094964 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d954a90-57b1-4d25-b734-f5ed93dfee3b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:00:11.096427 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d954a90-57b1-4d25-b734-f5ed93dfee3b None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:00:11.157802 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:00:11.198850 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 489/974] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Jul 13 22:00:10 2022] GET /identity/v3/auth/tokens => generated 1464 bytes in 315 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 13 22:00:11.205138 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b859fec5-723a-46a7-a872-536c1ab2726b None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:00:11.307370 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b859fec5-723a-46a7-a872-536c1ab2726b None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:00:11.341453 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6984b45-b563-4ce3-a2d4-5a838a079cd8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:00:11.342260 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6984b45-b563-4ce3-a2d4-5a838a079cd8 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:00:11.342567 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6984b45-b563-4ce3-a2d4-5a838a079cd8 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:00:11.342844 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6984b45-b563-4ce3-a2d4-5a838a079cd8 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:00:11.353358 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6984b45-b563-4ce3-a2d4-5a838a079cd8 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:00:11.354836 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6984b45-b563-4ce3-a2d4-5a838a079cd8 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:00:11.373863 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application [None req-a6984b45-b563-4ce3-a2d4-5a838a079cd8 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 22:00:11.373863 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 22:00:11.373863 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 22:00:11.373863 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 22:00:11.373863 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 22:00:11.373863 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 22:00:11.373863 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 22:00:11.373863 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 22:00:11.373863 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 22:00:11.373863 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 22:00:11.373863 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 22:00:11.373863 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 22:00:11.373863 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 22:00:11.373863 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 22:00:11.373863 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 22:00:11.374819 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 22:00:11.374819 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 22:00:11.374819 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 22:00:11.374819 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 22:00:11.374819 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 22:00:11.374819 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 22:00:11.374819 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 22:00:11.374819 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 22:00:11.374819 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application with Lock( Jul 13 22:00:11.374819 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 22:00:11.374819 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._enter() Jul 13 22:00:11.374819 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 22:00:11.374819 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 22:00:11.374819 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 22:00:11.374819 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.creator() Jul 13 22:00:11.374819 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 22:00:11.374819 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application created_value = creator( Jul 13 22:00:11.375739 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 22:00:11.375739 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 22:00:11.375739 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 22:00:11.375739 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Jul 13 22:00:11.375739 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 486/975] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Jul 13 22:00:11 2022] GET /identity/v3/domains/Default => generated 87 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 22:00:11.381791 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d954a90-57b1-4d25-b734-f5ed93dfee3b None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:00:11.484244 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d954a90-57b1-4d25-b734-f5ed93dfee3b None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:00:11.522678 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce94aa47-ef4a-4747-abd2-9bad6dda90bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:00:11.523345 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce94aa47-ef4a-4747-abd2-9bad6dda90bd None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:00:11.523634 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce94aa47-ef4a-4747-abd2-9bad6dda90bd None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:00:11.523912 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce94aa47-ef4a-4747-abd2-9bad6dda90bd None admin] PATH_INFO: `/v3/domains` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:00:11.524643 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce94aa47-ef4a-4747-abd2-9bad6dda90bd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 22:00:11.524942 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce94aa47-ef4a-4747-abd2-9bad6dda90bd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:00:11.526325 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce94aa47-ef4a-4747-abd2-9bad6dda90bd None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:00:11.542528 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 490/976] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Jul 13 22:00:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 22:00:11.546831 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6984b45-b563-4ce3-a2d4-5a838a079cd8 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:00:11.568082 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:00:11.580780 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f93f997-b0e4-4fc0-9c88-ab5003e89abe None None] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:00:11.581064 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f93f997-b0e4-4fc0-9c88-ab5003e89abe None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:00:11.581340 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f93f997-b0e4-4fc0-9c88-ab5003e89abe None None] PATH_INFO: `/` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:00:11.582785 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 491/977] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jul 13 22:00:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 22:00:11.587827 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f2d3f3-557d-4bfb-b1c4-cda15672dc25 None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:00:11.588108 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f2d3f3-557d-4bfb-b1c4-cda15672dc25 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:00:11.588378 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56f2d3f3-557d-4bfb-b1c4-cda15672dc25 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:00:11.606406 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-56f2d3f3-557d-4bfb-b1c4-cda15672dc25 None None] MFA Rules not processed for user `cc99c93da5074cf5a48bc2aa9a92c5e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 22:00:11.607625 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-56f2d3f3-557d-4bfb-b1c4-cda15672dc25 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:00:11.625415 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:00:11.662048 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6984b45-b563-4ce3-a2d4-5a838a079cd8 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:00:11.675948 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 492/978] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jul 13 22:00:11 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 13 22:00:11.677152 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe659ab-5aee-4359-a12e-31d199a4ff59 None None] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:00:11.677440 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe659ab-5aee-4359-a12e-31d199a4ff59 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:00:11.677713 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efe659ab-5aee-4359-a12e-31d199a4ff59 None None] PATH_INFO: `/` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:00:11.679253 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 493/979] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jul 13 22:00:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 22:00:11.684824 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05b5c31-c008-4374-8b4d-88f986214085 None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:00:11.685090 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05b5c31-c008-4374-8b4d-88f986214085 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:00:11.685827 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b05b5c31-c008-4374-8b4d-88f986214085 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:00:11.700158 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2890bd2a-c370-4681-a158-5f4f9a0c26b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:00:11.700970 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2890bd2a-c370-4681-a158-5f4f9a0c26b8 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:00:11.701254 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2890bd2a-c370-4681-a158-5f4f9a0c26b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:00:11.701527 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2890bd2a-c370-4681-a158-5f4f9a0c26b8 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:00:11.704300 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-b05b5c31-c008-4374-8b4d-88f986214085 None None] MFA Rules not processed for user `cc99c93da5074cf5a48bc2aa9a92c5e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 22:00:11.705482 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-b05b5c31-c008-4374-8b4d-88f986214085 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:00:11.708397 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2890bd2a-c370-4681-a158-5f4f9a0c26b8 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:00:11.709849 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2890bd2a-c370-4681-a158-5f4f9a0c26b8 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:00:11.723504 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application [None req-2890bd2a-c370-4681-a158-5f4f9a0c26b8 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 13 22:00:11.723504 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 22:00:11.723504 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 22:00:11.723504 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 22:00:11.723504 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 22:00:11.723504 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 22:00:11.723504 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 22:00:11.723504 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 22:00:11.723504 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 22:00:11.723504 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 22:00:11.723504 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 22:00:11.723504 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 22:00:11.723504 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 13 22:00:11.723504 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 13 22:00:11.723504 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 13 22:00:11.724865 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 13 22:00:11.724865 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 22:00:11.724865 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 22:00:11.724865 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 22:00:11.724865 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 22:00:11.724865 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 22:00:11.724865 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 22:00:11.724865 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 22:00:11.724865 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application with Lock( Jul 13 22:00:11.724865 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 22:00:11.724865 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._enter() Jul 13 22:00:11.724865 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 22:00:11.724865 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 22:00:11.724865 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 22:00:11.724865 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.creator() Jul 13 22:00:11.724865 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 22:00:11.724865 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application created_value = creator( Jul 13 22:00:11.725779 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 13 22:00:11.725779 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 13 22:00:11.725779 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 13 22:00:11.725779 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 13 22:00:11.725779 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 13 22:00:11.725779 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 13 22:00:11.725779 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 13 22:00:11.725779 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Jul 13 22:00:11.725779 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 487/980] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Jul 13 22:00:11 2022] GET /identity/v3/projects/service => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 22:00:11.725779 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2890bd2a-c370-4681-a158-5f4f9a0c26b8 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:00:11.775319 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 494/981] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jul 13 22:00:11 2022] POST /identity/v3/auth/tokens => generated 1464 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 13 22:00:11.776081 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05b5c31-c008-4374-8b4d-88f986214085 None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:00:11.801428 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:00:11.826061 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2890bd2a-c370-4681-a158-5f4f9a0c26b8 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:00:11.858998 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ecff84-f2d1-4588-93b9-1c7a7a0dc666 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:00:11.859660 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ecff84-f2d1-4588-93b9-1c7a7a0dc666 None admin] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:00:11.859938 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ecff84-f2d1-4588-93b9-1c7a7a0dc666 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:00:11.860193 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94ecff84-f2d1-4588-93b9-1c7a7a0dc666 None admin] PATH_INFO: `/v3/services` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:00:11.860917 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ecff84-f2d1-4588-93b9-1c7a7a0dc666 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:00:11.862280 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94ecff84-f2d1-4588-93b9-1c7a7a0dc666 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:00:11.868419 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:00:11.878520 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 488/982] 10.222.0.52 () {60 vars in 1141 bytes} [Wed Jul 13 22:00:11 2022] POST /identity/v3/services => generated 242 bytes in 153 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 22:00:11.879238 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ecff84-f2d1-4588-93b9-1c7a7a0dc666 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:00:11.885241 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b05b5c31-c008-4374-8b4d-88f986214085 None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:00:11.931752 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5658996e-76fa-4f4f-bd62-5ef60d768799 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:00:11.932460 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5658996e-76fa-4f4f-bd62-5ef60d768799 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:00:11.932740 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5658996e-76fa-4f4f-bd62-5ef60d768799 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:00:11.933015 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5658996e-76fa-4f4f-bd62-5ef60d768799 None admin] PATH_INFO: `/v3/projects` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:00:11.933846 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5658996e-76fa-4f4f-bd62-5ef60d768799 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 22:00:11.934160 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5658996e-76fa-4f4f-bd62-5ef60d768799 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:00:11.935696 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5658996e-76fa-4f4f-bd62-5ef60d768799 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:00:11.948803 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 495/983] 10.222.0.52 () {56 vars in 1148 bytes} [Wed Jul 13 22:00:11 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 22:00:11.954961 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5658996e-76fa-4f4f-bd62-5ef60d768799 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:00:12.003227 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94ecff84-f2d1-4588-93b9-1c7a7a0dc666 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:00:12.042276 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfdb0d90-7ff6-4383-b223-73add4cdc150 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:00:12.043075 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdb0d90-7ff6-4383-b223-73add4cdc150 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:00:12.043356 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdb0d90-7ff6-4383-b223-73add4cdc150 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:00:12.043628 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cfdb0d90-7ff6-4383-b223-73add4cdc150 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:00:12.044318 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfdb0d90-7ff6-4383-b223-73add4cdc150 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:00:12.045719 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cfdb0d90-7ff6-4383-b223-73add4cdc150 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:00:12.047902 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:00:12.060500 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application [None req-cfdb0d90-7ff6-4383-b223-73add4cdc150 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 13 22:00:12.060500 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 22:00:12.060500 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 13 22:00:12.060500 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 13 22:00:12.060500 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 13 22:00:12.060500 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 13 22:00:12.060500 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 13 22:00:12.060500 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 13 22:00:12.060500 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 13 22:00:12.060500 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Jul 13 22:00:12.060500 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 13 22:00:12.060500 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Jul 13 22:00:12.060500 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 22:00:12.060500 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 22:00:12.060500 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 22:00:12.060500 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 22:00:12.060500 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 22:00:12.061797 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 22:00:12.061797 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 22:00:12.061797 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 22:00:12.061797 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 22:00:12.061797 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 22:00:12.061797 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 22:00:12.061797 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 13 22:00:12.061797 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 13 22:00:12.061797 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 13 22:00:12.061797 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 13 22:00:12.061797 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 22:00:12.061797 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 22:00:12.061797 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 22:00:12.061797 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 22:00:12.061797 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 22:00:12.061797 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 22:00:12.062960 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 22:00:12.062960 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application with Lock( Jul 13 22:00:12.062960 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 22:00:12.062960 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._enter() Jul 13 22:00:12.062960 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 22:00:12.062960 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 22:00:12.062960 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 22:00:12.062960 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.creator() Jul 13 22:00:12.062960 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 22:00:12.062960 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application created_value = creator( Jul 13 22:00:12.062960 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 13 22:00:12.062960 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 13 22:00:12.062960 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 13 22:00:12.062960 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Jul 13 22:00:12.065622 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 489/984] 10.222.0.52 () {56 vars in 1126 bytes} [Wed Jul 13 22:00:11 2022] GET /identity/v3/services/object-store => generated 93 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 22:00:12.072603 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfdb0d90-7ff6-4383-b223-73add4cdc150 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:00:12.074514 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5658996e-76fa-4f4f-bd62-5ef60d768799 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:00:12.108228 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00872f3c-e89f-4e0d-b6b3-18aa73e50112 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:00:12.108899 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00872f3c-e89f-4e0d-b6b3-18aa73e50112 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:00:12.109181 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00872f3c-e89f-4e0d-b6b3-18aa73e50112 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:00:12.109456 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00872f3c-e89f-4e0d-b6b3-18aa73e50112 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:00:12.110293 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00872f3c-e89f-4e0d-b6b3-18aa73e50112 None admin] RBAC: Adding query filter params (role.id=3987457f5fe64b568601d117c2c0ba9d, scope.project.id=417e3e2f1333481795a089bef2e24bea, user.id=b866bacdafae4db6a32714c07c56515f) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 22:00:12.110640 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00872f3c-e89f-4e0d-b6b3-18aa73e50112 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:00:12.112083 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00872f3c-e89f-4e0d-b6b3-18aa73e50112 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:00:12.118409 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 496/985] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Jul 13 22:00:11 2022] GET /identity/v3/role_assignments?user.id=b866bacdafae4db6a32714c07c56515f&scope.project.id=417e3e2f1333481795a089bef2e24bea&role.id=3987457f5fe64b568601d117c2c0ba9d => generated 608 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 22:00:12.179760 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cfdb0d90-7ff6-4383-b223-73add4cdc150 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:00:12.218959 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d53a987-dfae-4d5c-997d-b85759637316 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:00:12.219645 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d53a987-dfae-4d5c-997d-b85759637316 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:00:12.219931 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d53a987-dfae-4d5c-997d-b85759637316 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:00:12.220214 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d53a987-dfae-4d5c-997d-b85759637316 None admin] PATH_INFO: `/v3/services` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:00:12.220941 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d53a987-dfae-4d5c-997d-b85759637316 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 22:00:12.221249 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d53a987-dfae-4d5c-997d-b85759637316 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:00:12.222669 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d53a987-dfae-4d5c-997d-b85759637316 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:00:12.228681 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 490/986] 10.222.0.52 () {56 vars in 1122 bytes} [Wed Jul 13 22:00:12 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 22:00:12.234461 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00872f3c-e89f-4e0d-b6b3-18aa73e50112 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:00:12.238949 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:00:12.355596 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00872f3c-e89f-4e0d-b6b3-18aa73e50112 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:00:12.392038 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0354c9b1-3809-49ac-8e4c-e80390bfe492 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:00:12.392714 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0354c9b1-3809-49ac-8e4c-e80390bfe492 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:00:12.392993 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0354c9b1-3809-49ac-8e4c-e80390bfe492 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:00:12.393265 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0354c9b1-3809-49ac-8e4c-e80390bfe492 None admin] PATH_INFO: `/v3/services` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:00:12.393969 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0354c9b1-3809-49ac-8e4c-e80390bfe492 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 22:00:12.394270 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0354c9b1-3809-49ac-8e4c-e80390bfe492 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:00:12.395684 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0354c9b1-3809-49ac-8e4c-e80390bfe492 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:00:12.401675 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 497/987] 10.222.0.52 () {56 vars in 1122 bytes} [Wed Jul 13 22:00:12 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 168 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 22:00:12.407571 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d53a987-dfae-4d5c-997d-b85759637316 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:00:12.473981 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:00:12.516416 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d53a987-dfae-4d5c-997d-b85759637316 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:00:12.557576 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b57ca615-fac6-445a-8903-2b2083df5064 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:00:12.558254 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57ca615-fac6-445a-8903-2b2083df5064 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:00:12.558525 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57ca615-fac6-445a-8903-2b2083df5064 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:00:12.558837 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b57ca615-fac6-445a-8903-2b2083df5064 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:00:12.559640 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b57ca615-fac6-445a-8903-2b2083df5064 None admin] RBAC: Adding query filter params (interface=admin, service_id=1b863c3a98f24d66a34f0bde03ae23a4, region_id=RegionOne) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 22:00:12.559953 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b57ca615-fac6-445a-8903-2b2083df5064 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:00:12.561338 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b57ca615-fac6-445a-8903-2b2083df5064 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:00:12.566970 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 491/988] 10.222.0.52 () {56 vars in 1249 bytes} [Wed Jul 13 22:00:12 2022] GET /identity/v3/endpoints?service_id=1b863c3a98f24d66a34f0bde03ae23a4&interface=admin®ion_id=RegionOne => generated 193 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 22:00:12.572624 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0354c9b1-3809-49ac-8e4c-e80390bfe492 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:00:12.643416 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:00:12.679478 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0354c9b1-3809-49ac-8e4c-e80390bfe492 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:00:12.714995 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5678b7d-7613-4b58-b637-4bf0308d5a2f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:00:12.715657 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5678b7d-7613-4b58-b637-4bf0308d5a2f None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:00:12.715932 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5678b7d-7613-4b58-b637-4bf0308d5a2f None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:00:12.716200 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5678b7d-7613-4b58-b637-4bf0308d5a2f None admin] PATH_INFO: `/v3/services` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:00:12.716875 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5678b7d-7613-4b58-b637-4bf0308d5a2f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:00:12.718248 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5678b7d-7613-4b58-b637-4bf0308d5a2f None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:00:12.724786 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 498/989] 10.222.0.52 () {56 vars in 1087 bytes} [Wed Jul 13 22:00:12 2022] GET /identity/v3/services => generated 1727 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 22:00:13.899485 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4344cfc2-36e1-45f9-9b94-f4cae06dc7d8 None None] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:00:13.900020 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4344cfc2-36e1-45f9-9b94-f4cae06dc7d8 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:00:13.900020 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4344cfc2-36e1-45f9-9b94-f4cae06dc7d8 None None] PATH_INFO: `/` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:00:13.901402 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 492/990] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jul 13 22:00:13 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 22:00:13.907041 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d220b1-cf78-484c-81c7-d1a18dc91277 None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:00:13.907460 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d220b1-cf78-484c-81c7-d1a18dc91277 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:00:13.907642 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64d220b1-cf78-484c-81c7-d1a18dc91277 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:00:13.925027 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-64d220b1-cf78-484c-81c7-d1a18dc91277 None None] MFA Rules not processed for user `cc99c93da5074cf5a48bc2aa9a92c5e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 22:00:13.926200 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-64d220b1-cf78-484c-81c7-d1a18dc91277 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:00:13.996578 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 499/991] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jul 13 22:00:13 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 13 22:00:14.019012 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4344cfc2-36e1-45f9-9b94-f4cae06dc7d8 None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:00:14.123943 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4344cfc2-36e1-45f9-9b94-f4cae06dc7d8 None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:00:14.148836 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:00:14.158591 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c7841b8-c936-4580-b535-808bc1a27b3c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:00:14.159355 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7841b8-c936-4580-b535-808bc1a27b3c None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:00:14.159703 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7841b8-c936-4580-b535-808bc1a27b3c None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:00:14.159905 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c7841b8-c936-4580-b535-808bc1a27b3c None admin] PATH_INFO: `/v3/services/compute` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:00:14.160555 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c7841b8-c936-4580-b535-808bc1a27b3c None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:00:14.161928 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0c7841b8-c936-4580-b535-808bc1a27b3c None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:00:14.173986 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application [None req-0c7841b8-c936-4580-b535-808bc1a27b3c None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 13 22:00:14.173986 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 22:00:14.173986 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 13 22:00:14.173986 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 13 22:00:14.173986 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 13 22:00:14.173986 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 13 22:00:14.173986 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 13 22:00:14.173986 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 13 22:00:14.173986 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 13 22:00:14.173986 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Jul 13 22:00:14.173986 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 13 22:00:14.173986 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Jul 13 22:00:14.173986 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 22:00:14.173986 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 22:00:14.173986 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 22:00:14.173986 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 22:00:14.173986 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 22:00:14.175190 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 22:00:14.175190 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 22:00:14.175190 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 22:00:14.175190 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 22:00:14.175190 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 22:00:14.175190 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 22:00:14.175190 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 13 22:00:14.175190 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 13 22:00:14.175190 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 13 22:00:14.175190 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 13 22:00:14.175190 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 22:00:14.175190 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 22:00:14.175190 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 22:00:14.175190 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 22:00:14.175190 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 22:00:14.175190 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 22:00:14.176051 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 22:00:14.176051 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application with Lock( Jul 13 22:00:14.176051 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 22:00:14.176051 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._enter() Jul 13 22:00:14.176051 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 22:00:14.176051 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 22:00:14.176051 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 22:00:14.176051 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.creator() Jul 13 22:00:14.176051 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 22:00:14.176051 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application created_value = creator( Jul 13 22:00:14.176051 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 13 22:00:14.176051 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 13 22:00:14.176051 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 13 22:00:14.176051 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Jul 13 22:00:14.177354 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 493/992] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Jul 13 22:00:14 2022] GET /identity/v3/services/compute => generated 88 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 22:00:14.181650 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d220b1-cf78-484c-81c7-d1a18dc91277 None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:00:14.198380 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49dcb54-724c-49d8-adda-9f6422966676 None None] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:00:14.198702 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49dcb54-724c-49d8-adda-9f6422966676 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:00:14.199309 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f49dcb54-724c-49d8-adda-9f6422966676 None None] PATH_INFO: `/` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:00:14.201265 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 494/993] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jul 13 22:00:14 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 22:00:14.206340 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0216496-af87-45af-9398-4bc8ebbd583f None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:00:14.206625 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0216496-af87-45af-9398-4bc8ebbd583f None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:00:14.206901 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0216496-af87-45af-9398-4bc8ebbd583f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:00:14.224347 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-b0216496-af87-45af-9398-4bc8ebbd583f None None] MFA Rules not processed for user `cc99c93da5074cf5a48bc2aa9a92c5e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 22:00:14.225896 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-b0216496-af87-45af-9398-4bc8ebbd583f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:00:14.248545 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:00:14.291624 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64d220b1-cf78-484c-81c7-d1a18dc91277 None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:00:14.295189 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 495/994] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jul 13 22:00:14 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 13 22:00:14.317097 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0216496-af87-45af-9398-4bc8ebbd583f None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:00:14.329204 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6c7237-a9b2-483b-8644-29a59d76b364 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:00:14.329884 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6c7237-a9b2-483b-8644-29a59d76b364 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:00:14.330167 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6c7237-a9b2-483b-8644-29a59d76b364 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:00:14.330437 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c6c7237-a9b2-483b-8644-29a59d76b364 None admin] PATH_INFO: `/v3/services` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:00:14.331174 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6c7237-a9b2-483b-8644-29a59d76b364 None admin] RBAC: Adding query filter params (name=compute) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 22:00:14.331475 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6c7237-a9b2-483b-8644-29a59d76b364 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:00:14.332857 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c6c7237-a9b2-483b-8644-29a59d76b364 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:00:14.338585 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 500/995] 10.222.0.52 () {56 vars in 1112 bytes} [Wed Jul 13 22:00:14 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 22:00:14.344295 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6c7237-a9b2-483b-8644-29a59d76b364 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:00:14.421241 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0216496-af87-45af-9398-4bc8ebbd583f None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:00:14.442080 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:00:14.457663 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc16580e-5174-4752-827e-b46040bd3c33 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:00:14.457854 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c6c7237-a9b2-483b-8644-29a59d76b364 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:00:14.458468 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc16580e-5174-4752-827e-b46040bd3c33 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:00:14.458781 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc16580e-5174-4752-827e-b46040bd3c33 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:00:14.459057 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc16580e-5174-4752-827e-b46040bd3c33 None admin] PATH_INFO: `/v3/services/image` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:00:14.459713 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc16580e-5174-4752-827e-b46040bd3c33 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:00:14.461327 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc16580e-5174-4752-827e-b46040bd3c33 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:00:14.473119 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application [None req-cc16580e-5174-4752-827e-b46040bd3c33 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 13 22:00:14.473119 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 22:00:14.473119 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 13 22:00:14.473119 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 13 22:00:14.473119 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 13 22:00:14.473119 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 13 22:00:14.473119 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 13 22:00:14.473119 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 13 22:00:14.473119 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 13 22:00:14.473119 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Jul 13 22:00:14.473119 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 13 22:00:14.473119 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Jul 13 22:00:14.473119 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 22:00:14.473119 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 22:00:14.473119 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 22:00:14.473119 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 22:00:14.473119 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 22:00:14.474123 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 22:00:14.474123 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 22:00:14.474123 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 22:00:14.474123 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 22:00:14.474123 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 22:00:14.474123 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 22:00:14.474123 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 13 22:00:14.474123 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 13 22:00:14.474123 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 13 22:00:14.474123 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 13 22:00:14.474123 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 22:00:14.474123 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 22:00:14.474123 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 22:00:14.474123 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 22:00:14.474123 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 22:00:14.474123 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 22:00:14.474994 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 22:00:14.474994 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application with Lock( Jul 13 22:00:14.474994 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 22:00:14.474994 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._enter() Jul 13 22:00:14.474994 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 22:00:14.474994 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 22:00:14.474994 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 22:00:14.474994 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.creator() Jul 13 22:00:14.474994 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 22:00:14.474994 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application created_value = creator( Jul 13 22:00:14.474994 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 13 22:00:14.474994 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 13 22:00:14.474994 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 13 22:00:14.474994 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Jul 13 22:00:14.474994 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 496/996] 10.222.0.52 () {56 vars in 1105 bytes} [Wed Jul 13 22:00:14 2022] GET /identity/v3/services/image => generated 86 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 13 22:00:14.480320 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc16580e-5174-4752-827e-b46040bd3c33 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:00:14.491490 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:00:14.503468 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f36274-6a36-4617-81cf-e6f07a32fe06 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:00:14.504152 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f36274-6a36-4617-81cf-e6f07a32fe06 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:00:14.504693 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f36274-6a36-4617-81cf-e6f07a32fe06 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:00:14.504971 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f36274-6a36-4617-81cf-e6f07a32fe06 None admin] PATH_INFO: `/v3/services` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:00:14.505936 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f36274-6a36-4617-81cf-e6f07a32fe06 None admin] RBAC: Adding query filter params (type=compute) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 22:00:14.506572 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f36274-6a36-4617-81cf-e6f07a32fe06 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:00:14.508048 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f36274-6a36-4617-81cf-e6f07a32fe06 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:00:14.513420 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 501/997] 10.222.0.52 () {56 vars in 1112 bytes} [Wed Jul 13 22:00:14 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 22:00:14.518936 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f36274-6a36-4617-81cf-e6f07a32fe06 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:00:14.592447 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc16580e-5174-4752-827e-b46040bd3c33 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:00:14.628803 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7038b91-a48c-470b-9348-cddeb0ce3e25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:00:14.629779 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7038b91-a48c-470b-9348-cddeb0ce3e25 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:00:14.630262 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7038b91-a48c-470b-9348-cddeb0ce3e25 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:00:14.630833 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7038b91-a48c-470b-9348-cddeb0ce3e25 None admin] PATH_INFO: `/v3/services` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:00:14.631838 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7038b91-a48c-470b-9348-cddeb0ce3e25 None admin] RBAC: Adding query filter params (name=image) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 22:00:14.632146 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7038b91-a48c-470b-9348-cddeb0ce3e25 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:00:14.633566 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7038b91-a48c-470b-9348-cddeb0ce3e25 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:00:14.638291 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f36274-6a36-4617-81cf-e6f07a32fe06 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:00:14.639222 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 497/998] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Jul 13 22:00:14 2022] GET /identity/v3/services?name=image => generated 122 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 22:00:14.644856 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7038b91-a48c-470b-9348-cddeb0ce3e25 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:00:14.679285 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0c17ba-58d1-4d98-a43f-e9999b4fa258 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:00:14.679929 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0c17ba-58d1-4d98-a43f-e9999b4fa258 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:00:14.680232 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0c17ba-58d1-4d98-a43f-e9999b4fa258 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:00:14.680506 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce0c17ba-58d1-4d98-a43f-e9999b4fa258 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:00:14.681305 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0c17ba-58d1-4d98-a43f-e9999b4fa258 None admin] RBAC: Adding query filter params (interface=public, service_id=5d2b1671ede7470bbd6a0d86b2fe8fab, region_id=RegionOne) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 22:00:14.681617 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0c17ba-58d1-4d98-a43f-e9999b4fa258 None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:00:14.683031 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce0c17ba-58d1-4d98-a43f-e9999b4fa258 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:00:14.692688 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 502/999] 10.222.0.52 () {56 vars in 1251 bytes} [Wed Jul 13 22:00:14 2022] GET /identity/v3/endpoints?service_id=5d2b1671ede7470bbd6a0d86b2fe8fab&interface=public®ion_id=RegionOne => generated 194 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 22:00:14.693587 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0c17ba-58d1-4d98-a43f-e9999b4fa258 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:00:14.708594 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:00:14.751953 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:00:14.762043 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7038b91-a48c-470b-9348-cddeb0ce3e25 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:00:14.796272 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0dba9d6-d960-48c0-9246-5aa80dde5c0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:00:14.796501 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce0c17ba-58d1-4d98-a43f-e9999b4fa258 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:00:14.796937 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0dba9d6-d960-48c0-9246-5aa80dde5c0b None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:00:14.797214 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0dba9d6-d960-48c0-9246-5aa80dde5c0b None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:00:14.797488 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0dba9d6-d960-48c0-9246-5aa80dde5c0b None admin] PATH_INFO: `/v3/services` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:00:14.798195 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0dba9d6-d960-48c0-9246-5aa80dde5c0b None admin] RBAC: Adding query filter params (type=image) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 22:00:14.798521 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0dba9d6-d960-48c0-9246-5aa80dde5c0b None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:00:14.799903 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0dba9d6-d960-48c0-9246-5aa80dde5c0b None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:00:14.804535 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 498/1000] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Jul 13 22:00:14 2022] GET /identity/v3/services?type=image => generated 122 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 22:00:14.806915 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dada7945-1b85-4450-8269-4a494088ecd6 None None] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:00:14.807193 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dada7945-1b85-4450-8269-4a494088ecd6 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:00:14.807487 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dada7945-1b85-4450-8269-4a494088ecd6 None None] PATH_INFO: `/` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:00:14.809012 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 499/1001] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jul 13 22:00:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 22:00:14.824685 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e89422-7f5c-43f8-9bec-55bf36730c95 None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:00:14.825010 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e89422-7f5c-43f8-9bec-55bf36730c95 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:00:14.825262 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4e89422-7f5c-43f8-9bec-55bf36730c95 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:00:14.833729 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a3facb-af7f-4795-8fee-1a2f8039034d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:00:14.834474 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a3facb-af7f-4795-8fee-1a2f8039034d None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:00:14.834794 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a3facb-af7f-4795-8fee-1a2f8039034d None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:00:14.835064 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73a3facb-af7f-4795-8fee-1a2f8039034d None admin] PATH_INFO: `/v3/services` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:00:14.835753 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a3facb-af7f-4795-8fee-1a2f8039034d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:00:14.838187 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73a3facb-af7f-4795-8fee-1a2f8039034d None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:00:14.843524 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-f4e89422-7f5c-43f8-9bec-55bf36730c95 None None] MFA Rules not processed for user `cc99c93da5074cf5a48bc2aa9a92c5e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 22:00:14.844758 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-f4e89422-7f5c-43f8-9bec-55bf36730c95 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:00:14.850731 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 503/1002] 10.222.0.52 () {56 vars in 1087 bytes} [Wed Jul 13 22:00:14 2022] GET /identity/v3/services => generated 1727 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 13 22:00:14.927619 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 500/1003] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jul 13 22:00:14 2022] POST /identity/v3/auth/tokens => generated 1560 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 13 22:00:14.952619 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a3facb-af7f-4795-8fee-1a2f8039034d None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:00:15.080208 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73a3facb-af7f-4795-8fee-1a2f8039034d None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:00:15.098059 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:00:15.115316 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57543823-bb94-4088-9d6f-0446f8b9fbe9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:00:15.116131 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57543823-bb94-4088-9d6f-0446f8b9fbe9 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:00:15.116405 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57543823-bb94-4088-9d6f-0446f8b9fbe9 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:00:15.116670 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57543823-bb94-4088-9d6f-0446f8b9fbe9 None admin] PATH_INFO: `/v3/services/object-store` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:00:15.117350 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57543823-bb94-4088-9d6f-0446f8b9fbe9 None admin] RBAC: Authorizing `identity:get_service(service_id=object-store)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:00:15.118752 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-57543823-bb94-4088-9d6f-0446f8b9fbe9 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:00:15.130947 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application [None req-57543823-bb94-4088-9d6f-0446f8b9fbe9 None admin] Could not find service: object-store.: keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 13 22:00:15.130947 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 22:00:15.130947 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 13 22:00:15.130947 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 13 22:00:15.130947 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 13 22:00:15.130947 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 13 22:00:15.130947 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 13 22:00:15.130947 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 13 22:00:15.130947 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 13 22:00:15.130947 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Jul 13 22:00:15.130947 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 13 22:00:15.130947 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Jul 13 22:00:15.130947 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 22:00:15.130947 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 22:00:15.130947 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 22:00:15.130947 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 22:00:15.130947 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 22:00:15.131972 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 22:00:15.131972 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 22:00:15.131972 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 22:00:15.131972 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 22:00:15.131972 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 22:00:15.131972 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 22:00:15.131972 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 13 22:00:15.131972 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 13 22:00:15.131972 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 13 22:00:15.131972 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 13 22:00:15.131972 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 22:00:15.131972 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 22:00:15.131972 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 22:00:15.131972 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 22:00:15.131972 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 22:00:15.131972 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 22:00:15.132839 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 22:00:15.132839 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application with Lock( Jul 13 22:00:15.132839 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 22:00:15.132839 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._enter() Jul 13 22:00:15.132839 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 22:00:15.132839 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 22:00:15.132839 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 22:00:15.132839 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.creator() Jul 13 22:00:15.132839 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 22:00:15.132839 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application created_value = creator( Jul 13 22:00:15.132839 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 13 22:00:15.132839 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 13 22:00:15.132839 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: object-store. Jul 13 22:00:15.132839 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Jul 13 22:00:15.132839 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 504/1004] 10.222.0.52 () {56 vars in 1126 bytes} [Wed Jul 13 22:00:14 2022] GET /identity/v3/services/object-store => generated 93 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 13 22:00:15.138242 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e89422-7f5c-43f8-9bec-55bf36730c95 None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:00:15.186701 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:00:15.247451 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4e89422-7f5c-43f8-9bec-55bf36730c95 None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:00:15.283224 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f257cc-c452-41ba-a35d-bfe6f6a29ab9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:00:15.283896 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f257cc-c452-41ba-a35d-bfe6f6a29ab9 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:00:15.284170 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f257cc-c452-41ba-a35d-bfe6f6a29ab9 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:00:15.284438 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f257cc-c452-41ba-a35d-bfe6f6a29ab9 None admin] PATH_INFO: `/v3/services` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:00:15.285152 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f257cc-c452-41ba-a35d-bfe6f6a29ab9 None admin] RBAC: Adding query filter params (name=object-store) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 22:00:15.285453 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f257cc-c452-41ba-a35d-bfe6f6a29ab9 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:00:15.286895 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f257cc-c452-41ba-a35d-bfe6f6a29ab9 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:00:15.298021 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 501/1005] 10.222.0.52 () {56 vars in 1122 bytes} [Wed Jul 13 22:00:15 2022] GET /identity/v3/services?name=object-store => generated 129 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 22:00:15.299406 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57543823-bb94-4088-9d6f-0446f8b9fbe9 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:00:15.409086 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57543823-bb94-4088-9d6f-0446f8b9fbe9 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:00:15.443810 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c807840d-2c44-41ac-b039-f6e605c4bba8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:00:15.444471 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c807840d-2c44-41ac-b039-f6e605c4bba8 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:00:15.444721 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c807840d-2c44-41ac-b039-f6e605c4bba8 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:00:15.445066 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c807840d-2c44-41ac-b039-f6e605c4bba8 None admin] PATH_INFO: `/v3/services` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:00:15.445866 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c807840d-2c44-41ac-b039-f6e605c4bba8 None admin] RBAC: Adding query filter params (type=object-store) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 22:00:15.446168 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c807840d-2c44-41ac-b039-f6e605c4bba8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:00:15.447736 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c807840d-2c44-41ac-b039-f6e605c4bba8 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:00:15.458022 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 505/1006] 10.222.0.52 () {56 vars in 1122 bytes} [Wed Jul 13 22:00:15 2022] GET /identity/v3/services?type=object-store => generated 356 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 22:00:15.459339 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f257cc-c452-41ba-a35d-bfe6f6a29ab9 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:00:15.567762 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f257cc-c452-41ba-a35d-bfe6f6a29ab9 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:00:15.576352 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:00:15.603902 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a8e316a-b710-49e7-96c6-f3e57ff2b145 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:00:15.604575 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8e316a-b710-49e7-96c6-f3e57ff2b145 None admin] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:00:15.604854 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8e316a-b710-49e7-96c6-f3e57ff2b145 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:00:15.605126 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a8e316a-b710-49e7-96c6-f3e57ff2b145 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:00:15.605869 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8e316a-b710-49e7-96c6-f3e57ff2b145 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:00:15.607266 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a8e316a-b710-49e7-96c6-f3e57ff2b145 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:00:15.624983 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 502/1007] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Jul 13 22:00:15 2022] POST /identity/v3/endpoints => generated 325 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 22:00:16.827689 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57251f2c-30d8-4776-9108-cc56140e2584 None None] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:00:16.828215 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57251f2c-30d8-4776-9108-cc56140e2584 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:00:16.828215 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57251f2c-30d8-4776-9108-cc56140e2584 None None] PATH_INFO: `/` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:00:16.829495 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 506/1008] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jul 13 22:00:16 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 22:00:16.835112 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e0904b-ae64-46ac-ae14-e3f8f9651958 None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:00:16.835530 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e0904b-ae64-46ac-ae14-e3f8f9651958 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:00:16.835702 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2e0904b-ae64-46ac-ae14-e3f8f9651958 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:00:16.859233 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-c2e0904b-ae64-46ac-ae14-e3f8f9651958 None None] MFA Rules not processed for user `cc99c93da5074cf5a48bc2aa9a92c5e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 22:00:16.860412 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-c2e0904b-ae64-46ac-ae14-e3f8f9651958 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:00:16.905295 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f376d3-cf07-432d-9d62-576d227b6b47 None None] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:00:16.905602 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f376d3-cf07-432d-9d62-576d227b6b47 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:00:16.905875 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96f376d3-cf07-432d-9d62-576d227b6b47 None None] PATH_INFO: `/` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:00:16.907947 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 507/1009] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jul 13 22:00:16 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 22:00:16.912774 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c164dfb-380f-4920-95c3-8737f0d9c092 None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:00:16.913036 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c164dfb-380f-4920-95c3-8737f0d9c092 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:00:16.913572 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c164dfb-380f-4920-95c3-8737f0d9c092 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:00:16.931856 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-1c164dfb-380f-4920-95c3-8737f0d9c092 None None] MFA Rules not processed for user `cc99c93da5074cf5a48bc2aa9a92c5e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 22:00:16.933069 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-1c164dfb-380f-4920-95c3-8737f0d9c092 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:00:16.938530 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 503/1010] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jul 13 22:00:16 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 13 22:00:16.960532 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e0904b-ae64-46ac-ae14-e3f8f9651958 None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:00:16.979073 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:00:17.010525 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 508/1011] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jul 13 22:00:16 2022] POST /identity/v3/auth/tokens => generated 1707 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 13 22:00:17.029487 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c164dfb-380f-4920-95c3-8737f0d9c092 None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:00:17.055290 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:00:17.064669 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2e0904b-ae64-46ac-ae14-e3f8f9651958 None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:00:17.101976 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b905a4-c8b2-431e-94e3-146556836175 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:00:17.102598 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b905a4-c8b2-431e-94e3-146556836175 None admin] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:00:17.102877 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b905a4-c8b2-431e-94e3-146556836175 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:00:17.103151 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36b905a4-c8b2-431e-94e3-146556836175 None admin] PATH_INFO: `/v3/services` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:00:17.103901 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b905a4-c8b2-431e-94e3-146556836175 None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:00:17.105290 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36b905a4-c8b2-431e-94e3-146556836175 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:00:17.119940 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 504/1012] 10.222.0.52 () {60 vars in 1141 bytes} [Wed Jul 13 22:00:16 2022] POST /identity/v3/services => generated 242 bytes in 160 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 22:00:17.138786 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c164dfb-380f-4920-95c3-8737f0d9c092 None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:00:17.184972 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78e6c33-cd92-4105-b6fa-4987c7ddd31b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:00:17.185794 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78e6c33-cd92-4105-b6fa-4987c7ddd31b None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:00:17.186071 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78e6c33-cd92-4105-b6fa-4987c7ddd31b None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:00:17.186340 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c78e6c33-cd92-4105-b6fa-4987c7ddd31b None admin] PATH_INFO: `/v3/services/compute` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:00:17.187026 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c78e6c33-cd92-4105-b6fa-4987c7ddd31b None admin] RBAC: Authorizing `identity:get_service(service_id=compute)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:00:17.188436 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c78e6c33-cd92-4105-b6fa-4987c7ddd31b None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:00:17.201706 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application [None req-c78e6c33-cd92-4105-b6fa-4987c7ddd31b None admin] Could not find service: compute.: keystone.exception.ServiceNotFound: Could not find service: compute. Jul 13 22:00:17.201706 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 22:00:17.201706 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 13 22:00:17.201706 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 13 22:00:17.201706 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 13 22:00:17.201706 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 13 22:00:17.201706 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 13 22:00:17.201706 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 13 22:00:17.201706 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 13 22:00:17.201706 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Jul 13 22:00:17.201706 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 13 22:00:17.201706 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Jul 13 22:00:17.201706 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 22:00:17.201706 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 22:00:17.201706 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 22:00:17.201706 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 22:00:17.201706 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 22:00:17.202913 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 22:00:17.202913 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 22:00:17.202913 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 22:00:17.202913 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 22:00:17.202913 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 22:00:17.202913 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 22:00:17.202913 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 13 22:00:17.202913 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 13 22:00:17.202913 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 13 22:00:17.202913 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 13 22:00:17.202913 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 22:00:17.202913 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 22:00:17.202913 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 22:00:17.202913 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 22:00:17.202913 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 22:00:17.202913 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 22:00:17.203915 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 22:00:17.203915 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application with Lock( Jul 13 22:00:17.203915 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 22:00:17.203915 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._enter() Jul 13 22:00:17.203915 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 22:00:17.203915 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 22:00:17.203915 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 22:00:17.203915 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.creator() Jul 13 22:00:17.203915 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 22:00:17.203915 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application created_value = creator( Jul 13 22:00:17.203915 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 13 22:00:17.203915 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 13 22:00:17.203915 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: compute. Jul 13 22:00:17.203915 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Jul 13 22:00:17.206896 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 509/1013] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Jul 13 22:00:17 2022] GET /identity/v3/services/compute => generated 88 bytes in 178 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 13 22:00:17.209285 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b905a4-c8b2-431e-94e3-146556836175 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:00:17.329358 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36b905a4-c8b2-431e-94e3-146556836175 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:00:17.364583 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f02b37-fdb6-4305-9715-9b3bc3ea7c6c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:00:17.365171 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f02b37-fdb6-4305-9715-9b3bc3ea7c6c None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:00:17.365451 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f02b37-fdb6-4305-9715-9b3bc3ea7c6c None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:00:17.365725 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30f02b37-fdb6-4305-9715-9b3bc3ea7c6c None admin] PATH_INFO: `/v3/services` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:00:17.366439 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f02b37-fdb6-4305-9715-9b3bc3ea7c6c None admin] RBAC: Adding query filter params (name=compute) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 22:00:17.366774 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f02b37-fdb6-4305-9715-9b3bc3ea7c6c None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:00:17.368187 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30f02b37-fdb6-4305-9715-9b3bc3ea7c6c None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:00:17.373373 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 505/1014] 10.222.0.52 () {56 vars in 1112 bytes} [Wed Jul 13 22:00:17 2022] GET /identity/v3/services?name=compute => generated 124 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 22:00:17.379294 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78e6c33-cd92-4105-b6fa-4987c7ddd31b None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:00:17.418565 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:00:17.485394 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c78e6c33-cd92-4105-b6fa-4987c7ddd31b None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:00:17.520340 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c86918f-df65-46af-b365-0cc6844bd58a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:00:17.521028 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c86918f-df65-46af-b365-0cc6844bd58a None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:00:17.521309 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c86918f-df65-46af-b365-0cc6844bd58a None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:00:17.521587 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c86918f-df65-46af-b365-0cc6844bd58a None admin] PATH_INFO: `/v3/services` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:00:17.522283 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c86918f-df65-46af-b365-0cc6844bd58a None admin] RBAC: Adding query filter params (type=compute) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 22:00:17.522634 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c86918f-df65-46af-b365-0cc6844bd58a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:00:17.524024 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c86918f-df65-46af-b365-0cc6844bd58a None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:00:17.529281 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 510/1015] 10.222.0.52 () {56 vars in 1112 bytes} [Wed Jul 13 22:00:17 2022] GET /identity/v3/services?type=compute => generated 352 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 22:00:17.535056 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f02b37-fdb6-4305-9715-9b3bc3ea7c6c None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:00:17.557268 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:00:17.627896 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b3ffdc-a5c2-435c-9314-77dd47aeeaae None None] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:00:17.628181 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b3ffdc-a5c2-435c-9314-77dd47aeeaae None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:00:17.628520 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61b3ffdc-a5c2-435c-9314-77dd47aeeaae None None] PATH_INFO: `/` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:00:17.630094 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 511/1016] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jul 13 22:00:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 13 22:00:17.635506 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f35d00-012e-4351-beb4-6530cfcf8cf6 None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:00:17.635506 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f35d00-012e-4351-beb4-6530cfcf8cf6 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:00:17.635797 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f35d00-012e-4351-beb4-6530cfcf8cf6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:00:17.644281 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30f02b37-fdb6-4305-9715-9b3bc3ea7c6c None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:00:17.652417 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-a6f35d00-012e-4351-beb4-6530cfcf8cf6 None None] MFA Rules not processed for user `cc99c93da5074cf5a48bc2aa9a92c5e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 22:00:17.653611 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-a6f35d00-012e-4351-beb4-6530cfcf8cf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:00:17.679102 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf9a1a2-b73c-4413-b168-dade472ade53 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:00:17.679859 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf9a1a2-b73c-4413-b168-dade472ade53 None admin] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:00:17.680142 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf9a1a2-b73c-4413-b168-dade472ade53 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:00:17.680613 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8bf9a1a2-b73c-4413-b168-dade472ade53 None admin] PATH_INFO: `/v3/endpoints` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:00:17.681256 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf9a1a2-b73c-4413-b168-dade472ade53 None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:00:17.682657 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8bf9a1a2-b73c-4413-b168-dade472ade53 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:00:17.705080 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 506/1017] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Jul 13 22:00:17 2022] POST /identity/v3/endpoints => generated 334 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 22:00:17.732092 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 512/1018] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jul 13 22:00:17 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 13 22:00:17.755775 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf9a1a2-b73c-4413-b168-dade472ade53 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:00:17.833599 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:00:17.872779 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8bf9a1a2-b73c-4413-b168-dade472ade53 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:00:17.910053 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9dccd556-2038-4655-8fdd-28e3e75073e0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:00:17.910899 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dccd556-2038-4655-8fdd-28e3e75073e0 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:00:17.911176 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dccd556-2038-4655-8fdd-28e3e75073e0 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:00:17.911441 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dccd556-2038-4655-8fdd-28e3e75073e0 None admin] PATH_INFO: `/v3/domains/default` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:00:17.912619 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dccd556-2038-4655-8fdd-28e3e75073e0 None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:00:17.914128 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9dccd556-2038-4655-8fdd-28e3e75073e0 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:00:17.916267 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 507/1019] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Jul 13 22:00:17 2022] GET /identity/v3/domains/default => generated 199 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 22:00:17.922434 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f35d00-012e-4351-beb4-6530cfcf8cf6 None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:00:17.951661 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:00:18.026860 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6f35d00-012e-4351-beb4-6530cfcf8cf6 None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:00:18.060748 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba684836-0fa9-407d-9fd5-2d8713a00bfe None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:00:18.061417 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba684836-0fa9-407d-9fd5-2d8713a00bfe None admin] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:00:18.061702 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba684836-0fa9-407d-9fd5-2d8713a00bfe None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:00:18.061975 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba684836-0fa9-407d-9fd5-2d8713a00bfe None admin] PATH_INFO: `/v3/projects` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:00:18.062730 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba684836-0fa9-407d-9fd5-2d8713a00bfe None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:00:18.064172 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba684836-0fa9-407d-9fd5-2d8713a00bfe None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:00:18.091735 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 513/1020] 10.222.0.52 () {60 vars in 1141 bytes} [Wed Jul 13 22:00:17 2022] POST /identity/v3/projects => generated 311 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 22:00:19.181211 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ece68ef-19f8-4d20-8f2e-e9fa415a153c None None] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:00:19.181798 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ece68ef-19f8-4d20-8f2e-e9fa415a153c None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:00:19.181798 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ece68ef-19f8-4d20-8f2e-e9fa415a153c None None] PATH_INFO: `/` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:00:19.183370 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 508/1021] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jul 13 22:00:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 22:00:19.188497 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a375bfa6-abea-4f13-bbd1-58de6609a964 None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:00:19.188938 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a375bfa6-abea-4f13-bbd1-58de6609a964 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:00:19.189032 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a375bfa6-abea-4f13-bbd1-58de6609a964 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:00:19.207534 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-a375bfa6-abea-4f13-bbd1-58de6609a964 None None] MFA Rules not processed for user `cc99c93da5074cf5a48bc2aa9a92c5e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 22:00:19.208722 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-a375bfa6-abea-4f13-bbd1-58de6609a964 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:00:19.281937 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 514/1022] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jul 13 22:00:19 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 13 22:00:19.305430 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ece68ef-19f8-4d20-8f2e-e9fa415a153c None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:00:19.409281 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ece68ef-19f8-4d20-8f2e-e9fa415a153c None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:00:19.443396 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e2f842-b07c-45e8-8561-03cc580845b9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:00:19.444117 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e2f842-b07c-45e8-8561-03cc580845b9 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:00:19.444397 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e2f842-b07c-45e8-8561-03cc580845b9 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:00:19.444669 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e2f842-b07c-45e8-8561-03cc580845b9 None admin] PATH_INFO: `/v3/services/image` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:00:19.445422 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e2f842-b07c-45e8-8561-03cc580845b9 None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:00:19.446859 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2e2f842-b07c-45e8-8561-03cc580845b9 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:00:19.459001 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application [None req-b2e2f842-b07c-45e8-8561-03cc580845b9 None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 13 22:00:19.459001 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 22:00:19.459001 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 13 22:00:19.459001 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 13 22:00:19.459001 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 13 22:00:19.459001 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 13 22:00:19.459001 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 13 22:00:19.459001 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 13 22:00:19.459001 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 13 22:00:19.459001 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Jul 13 22:00:19.459001 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 13 22:00:19.459001 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Jul 13 22:00:19.459001 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 22:00:19.459001 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 22:00:19.459001 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 22:00:19.459001 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 22:00:19.459001 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 22:00:19.459991 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 22:00:19.459991 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 22:00:19.459991 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 22:00:19.459991 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 22:00:19.459991 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 22:00:19.459991 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 22:00:19.459991 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 13 22:00:19.459991 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 13 22:00:19.459991 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 13 22:00:19.459991 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 13 22:00:19.459991 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 22:00:19.459991 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 22:00:19.459991 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 22:00:19.459991 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 22:00:19.459991 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 22:00:19.459991 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 22:00:19.460831 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 22:00:19.460831 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application with Lock( Jul 13 22:00:19.460831 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 22:00:19.460831 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._enter() Jul 13 22:00:19.460831 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 22:00:19.460831 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 22:00:19.460831 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 22:00:19.460831 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.creator() Jul 13 22:00:19.460831 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 22:00:19.460831 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application created_value = creator( Jul 13 22:00:19.460831 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 13 22:00:19.460831 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 13 22:00:19.460831 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 13 22:00:19.460831 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Jul 13 22:00:19.462822 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 509/1023] 10.222.0.52 () {56 vars in 1105 bytes} [Wed Jul 13 22:00:19 2022] GET /identity/v3/services/image => generated 86 bytes in 158 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 22:00:19.466273 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a375bfa6-abea-4f13-bbd1-58de6609a964 None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:00:19.477046 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:00:19.577952 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a375bfa6-abea-4f13-bbd1-58de6609a964 None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:00:19.614169 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ddd758-954e-4033-8f64-76e80729231a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:00:19.615247 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ddd758-954e-4033-8f64-76e80729231a None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:00:19.615752 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ddd758-954e-4033-8f64-76e80729231a None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:00:19.616235 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ddd758-954e-4033-8f64-76e80729231a None admin] PATH_INFO: `/v3/services` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:00:19.617180 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ddd758-954e-4033-8f64-76e80729231a None admin] RBAC: Adding query filter params (name=image) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 22:00:19.617737 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ddd758-954e-4033-8f64-76e80729231a None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:00:19.619389 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43ddd758-954e-4033-8f64-76e80729231a None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:00:19.629089 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 515/1024] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Jul 13 22:00:19 2022] GET /identity/v3/services?name=image => generated 122 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 22:00:19.630416 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e2f842-b07c-45e8-8561-03cc580845b9 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:00:19.635368 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:00:19.734711 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e2f842-b07c-45e8-8561-03cc580845b9 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:00:19.772971 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e2a3ba3-6fc1-40af-a676-09e0b95ceb41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:00:19.773930 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2a3ba3-6fc1-40af-a676-09e0b95ceb41 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:00:19.774214 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2a3ba3-6fc1-40af-a676-09e0b95ceb41 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:00:19.774506 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e2a3ba3-6fc1-40af-a676-09e0b95ceb41 None admin] PATH_INFO: `/v3/services` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:00:19.775601 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2a3ba3-6fc1-40af-a676-09e0b95ceb41 None admin] RBAC: Adding query filter params (type=image) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 22:00:19.775911 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2a3ba3-6fc1-40af-a676-09e0b95ceb41 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:00:19.777304 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e2a3ba3-6fc1-40af-a676-09e0b95ceb41 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:00:19.782900 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 510/1025] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Jul 13 22:00:19 2022] GET /identity/v3/services?type=image => generated 350 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 22:00:19.788748 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ddd758-954e-4033-8f64-76e80729231a None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:00:19.819424 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb373af3-c6e0-4ae4-a16d-47ea7757f884 None None] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:00:19.819752 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb373af3-c6e0-4ae4-a16d-47ea7757f884 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:00:19.819979 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb373af3-c6e0-4ae4-a16d-47ea7757f884 None None] PATH_INFO: `/` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:00:19.822243 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 511/1026] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jul 13 22:00:19 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 13 22:00:19.827688 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09904f18-f06d-41ee-a70a-04e3f580a821 None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:00:19.827977 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09904f18-f06d-41ee-a70a-04e3f580a821 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:00:19.828254 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09904f18-f06d-41ee-a70a-04e3f580a821 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:00:19.846902 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-09904f18-f06d-41ee-a70a-04e3f580a821 None None] MFA Rules not processed for user `cc99c93da5074cf5a48bc2aa9a92c5e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 22:00:19.848106 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-09904f18-f06d-41ee-a70a-04e3f580a821 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:00:19.871531 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:00:19.910440 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43ddd758-954e-4033-8f64-76e80729231a None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:00:19.911115 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:00:19.917711 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 512/1027] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jul 13 22:00:19 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 13 22:00:19.941536 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09904f18-f06d-41ee-a70a-04e3f580a821 None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:00:19.947532 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959f52a2-10f7-464f-9c3d-35d34bb7645f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:00:19.948207 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959f52a2-10f7-464f-9c3d-35d34bb7645f None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:00:19.948496 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959f52a2-10f7-464f-9c3d-35d34bb7645f None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:00:19.948767 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-959f52a2-10f7-464f-9c3d-35d34bb7645f None admin] PATH_INFO: `/v3/endpoints` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:00:19.949921 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-959f52a2-10f7-464f-9c3d-35d34bb7645f None admin] RBAC: Adding query filter params (interface=public, service_id=03087deb4dc2485db2bc6dfc3a8eb0ba, region_id=RegionOne) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 22:00:19.950232 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-959f52a2-10f7-464f-9c3d-35d34bb7645f None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:00:19.951659 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-959f52a2-10f7-464f-9c3d-35d34bb7645f None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:00:19.957331 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 516/1028] 10.222.0.52 () {56 vars in 1251 bytes} [Wed Jul 13 22:00:19 2022] GET /identity/v3/endpoints?service_id=03087deb4dc2485db2bc6dfc3a8eb0ba&interface=public®ion_id=RegionOne => generated 194 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 22:00:19.962875 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959f52a2-10f7-464f-9c3d-35d34bb7645f None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:00:20.049374 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09904f18-f06d-41ee-a70a-04e3f580a821 None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:00:20.067169 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-959f52a2-10f7-464f-9c3d-35d34bb7645f None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:00:20.086470 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1c73fe-bc9d-4070-b60f-456e84df41db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:00:20.087462 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1c73fe-bc9d-4070-b60f-456e84df41db None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:00:20.087740 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1c73fe-bc9d-4070-b60f-456e84df41db None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:00:20.088005 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce1c73fe-bc9d-4070-b60f-456e84df41db None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:00:20.099288 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce1c73fe-bc9d-4070-b60f-456e84df41db None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 22:00:20.102424 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1bd948d-cec3-4dbd-99bc-5c4fc029c1ff None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:00:20.103125 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bd948d-cec3-4dbd-99bc-5c4fc029c1ff None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:00:20.103410 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bd948d-cec3-4dbd-99bc-5c4fc029c1ff None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:00:20.103683 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1bd948d-cec3-4dbd-99bc-5c4fc029c1ff None admin] PATH_INFO: `/v3/services` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:00:20.104370 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1bd948d-cec3-4dbd-99bc-5c4fc029c1ff None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:00:20.105747 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b1bd948d-cec3-4dbd-99bc-5c4fc029c1ff None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:00:20.107553 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:00:20.109686 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application [None req-ce1c73fe-bc9d-4070-b60f-456e84df41db None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 13 22:00:20.109686 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 22:00:20.109686 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 22:00:20.109686 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 22:00:20.109686 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 22:00:20.109686 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 22:00:20.109686 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 22:00:20.109686 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 22:00:20.109686 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 22:00:20.109686 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 22:00:20.109686 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 22:00:20.109686 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 22:00:20.109686 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 22:00:20.109686 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 22:00:20.109686 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 22:00:20.109686 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 22:00:20.110982 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 22:00:20.110982 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 22:00:20.110982 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 22:00:20.110982 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 22:00:20.110982 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 22:00:20.110982 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 22:00:20.110982 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 22:00:20.110982 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 22:00:20.110982 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 22:00:20.110982 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 22:00:20.110982 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 22:00:20.110982 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application with Lock( Jul 13 22:00:20.110982 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 22:00:20.110982 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._enter() Jul 13 22:00:20.110982 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 22:00:20.110982 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 22:00:20.111837 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 22:00:20.111837 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.creator() Jul 13 22:00:20.111837 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 22:00:20.111837 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application created_value = creator( Jul 13 22:00:20.111837 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 22:00:20.111837 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 22:00:20.111837 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 22:00:20.111837 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 22:00:20.111837 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 22:00:20.111837 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 22:00:20.111837 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 13 22:00:20.111837 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Jul 13 22:00:20.111837 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 513/1029] 10.222.0.52 () {56 vars in 1120 bytes} [Wed Jul 13 22:00:19 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 13 22:00:20.116511 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 517/1030] 10.222.0.52 () {56 vars in 1087 bytes} [Wed Jul 13 22:00:19 2022] GET /identity/v3/services => generated 1957 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 13 22:00:20.126536 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1c73fe-bc9d-4070-b60f-456e84df41db None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:00:20.146409 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bd8cde-37ad-4de9-88d8-9cf281b5d5b9 None None] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:00:20.146801 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bd8cde-37ad-4de9-88d8-9cf281b5d5b9 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:00:20.147003 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40bd8cde-37ad-4de9-88d8-9cf281b5d5b9 None None] PATH_INFO: `/` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:00:20.148744 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 518/1031] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jul 13 22:00:20 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 13 22:00:20.154392 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee414eab-5655-4c8d-ab1e-c66dff0909b7 None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:00:20.154722 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee414eab-5655-4c8d-ab1e-c66dff0909b7 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:00:20.154993 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee414eab-5655-4c8d-ab1e-c66dff0909b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:00:20.172725 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-ee414eab-5655-4c8d-ab1e-c66dff0909b7 None None] MFA Rules not processed for user `cc99c93da5074cf5a48bc2aa9a92c5e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 22:00:20.173964 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-ee414eab-5655-4c8d-ab1e-c66dff0909b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:00:20.193865 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:00:20.244985 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce1c73fe-bc9d-4070-b60f-456e84df41db None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:00:20.254951 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 519/1032] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jul 13 22:00:20 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 13 22:00:20.269678 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee414eab-5655-4c8d-ab1e-c66dff0909b7 None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:00:20.287958 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28307a16-c6e0-4ef9-bd6d-6d2b0cd83993 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:00:20.288642 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28307a16-c6e0-4ef9-bd6d-6d2b0cd83993 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:00:20.288924 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28307a16-c6e0-4ef9-bd6d-6d2b0cd83993 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:00:20.289900 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28307a16-c6e0-4ef9-bd6d-6d2b0cd83993 None admin] PATH_INFO: `/v3/roles` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:00:20.290757 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28307a16-c6e0-4ef9-bd6d-6d2b0cd83993 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 22:00:20.291031 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28307a16-c6e0-4ef9-bd6d-6d2b0cd83993 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:00:20.292468 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28307a16-c6e0-4ef9-bd6d-6d2b0cd83993 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:00:20.301649 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 514/1033] 10.222.0.52 () {56 vars in 1115 bytes} [Wed Jul 13 22:00:20 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 22:00:20.307496 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28307a16-c6e0-4ef9-bd6d-6d2b0cd83993 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:00:20.387458 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee414eab-5655-4c8d-ab1e-c66dff0909b7 None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:00:20.414344 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28307a16-c6e0-4ef9-bd6d-6d2b0cd83993 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:00:20.417507 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:00:20.424794 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c019ec02-4f52-489e-9d82-ccd7b3bef2af None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:00:20.425908 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c019ec02-4f52-489e-9d82-ccd7b3bef2af None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:00:20.426185 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c019ec02-4f52-489e-9d82-ccd7b3bef2af None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:00:20.426459 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c019ec02-4f52-489e-9d82-ccd7b3bef2af None admin] PATH_INFO: `/v3/domains/default` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:00:20.427665 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c019ec02-4f52-489e-9d82-ccd7b3bef2af None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:00:20.429135 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c019ec02-4f52-489e-9d82-ccd7b3bef2af None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:00:20.431527 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 520/1034] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Jul 13 22:00:20 2022] GET /identity/v3/domains/default => generated 199 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 22:00:20.436128 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:00:20.437271 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c019ec02-4f52-489e-9d82-ccd7b3bef2af None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:00:20.453932 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9327e4-a8cf-4a8f-93fa-b0be76f5ef11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:00:20.454952 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9327e4-a8cf-4a8f-93fa-b0be76f5ef11 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:00:20.455065 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9327e4-a8cf-4a8f-93fa-b0be76f5ef11 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:00:20.455306 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a9327e4-a8cf-4a8f-93fa-b0be76f5ef11 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:00:20.467440 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9327e4-a8cf-4a8f-93fa-b0be76f5ef11 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:00:20.468906 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a9327e4-a8cf-4a8f-93fa-b0be76f5ef11 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:00:20.489992 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application [None req-0a9327e4-a8cf-4a8f-93fa-b0be76f5ef11 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 22:00:20.489992 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 22:00:20.489992 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 22:00:20.489992 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 22:00:20.489992 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 22:00:20.489992 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 22:00:20.489992 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 22:00:20.489992 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 22:00:20.489992 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 22:00:20.489992 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 22:00:20.489992 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 22:00:20.489992 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 22:00:20.489992 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 22:00:20.489992 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 22:00:20.489992 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 22:00:20.491175 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 22:00:20.491175 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 22:00:20.491175 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 22:00:20.491175 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 22:00:20.491175 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 22:00:20.491175 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 22:00:20.491175 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 22:00:20.491175 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 22:00:20.491175 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application with Lock( Jul 13 22:00:20.491175 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 22:00:20.491175 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._enter() Jul 13 22:00:20.491175 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 22:00:20.491175 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 22:00:20.491175 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 22:00:20.491175 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.creator() Jul 13 22:00:20.491175 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 22:00:20.491175 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application created_value = creator( Jul 13 22:00:20.492112 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 22:00:20.492112 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 22:00:20.492112 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 22:00:20.492112 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Jul 13 22:00:20.492112 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 515/1035] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Jul 13 22:00:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 22:00:20.497389 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9327e4-a8cf-4a8f-93fa-b0be76f5ef11 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:00:20.552043 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c019ec02-4f52-489e-9d82-ccd7b3bef2af None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:00:20.588150 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3077284-1e5c-4430-82e2-57b3e2c73557 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:00:20.588918 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3077284-1e5c-4430-82e2-57b3e2c73557 None admin] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:00:20.589262 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3077284-1e5c-4430-82e2-57b3e2c73557 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:00:20.589600 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3077284-1e5c-4430-82e2-57b3e2c73557 None admin] PATH_INFO: `/v3/users` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:00:20.590391 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3077284-1e5c-4430-82e2-57b3e2c73557 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:00:20.591962 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3077284-1e5c-4430-82e2-57b3e2c73557 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:00:20.610574 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a9327e4-a8cf-4a8f-93fa-b0be76f5ef11 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:00:20.614371 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 521/1036] 10.222.0.52 () {60 vars in 1132 bytes} [Wed Jul 13 22:00:20 2022] POST /identity/v3/users => generated 285 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 22:00:20.654086 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed36a4e1-e8a6-4b4c-afa2-aad076b3136d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:00:20.654879 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed36a4e1-e8a6-4b4c-afa2-aad076b3136d None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:00:20.655158 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed36a4e1-e8a6-4b4c-afa2-aad076b3136d None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:00:20.655428 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed36a4e1-e8a6-4b4c-afa2-aad076b3136d None admin] PATH_INFO: `/v3/domains` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:00:20.656167 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed36a4e1-e8a6-4b4c-afa2-aad076b3136d None admin] RBAC: Adding query filter params (name=Default) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 22:00:20.656530 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed36a4e1-e8a6-4b4c-afa2-aad076b3136d None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:00:20.657896 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed36a4e1-e8a6-4b4c-afa2-aad076b3136d None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:00:20.672456 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 516/1037] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Jul 13 22:00:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 22:00:20.677937 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3077284-1e5c-4430-82e2-57b3e2c73557 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:00:20.740615 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:00:20.795613 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3077284-1e5c-4430-82e2-57b3e2c73557 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:00:20.832945 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b224d8-7516-4e06-84d2-c162f5a34cb1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:00:20.834093 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b224d8-7516-4e06-84d2-c162f5a34cb1 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:00:20.834640 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b224d8-7516-4e06-84d2-c162f5a34cb1 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:00:20.835133 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6b224d8-7516-4e06-84d2-c162f5a34cb1 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:00:20.843393 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b224d8-7516-4e06-84d2-c162f5a34cb1 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:00:20.844849 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6b224d8-7516-4e06-84d2-c162f5a34cb1 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:00:20.859016 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application [None req-b6b224d8-7516-4e06-84d2-c162f5a34cb1 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 13 22:00:20.859016 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 22:00:20.859016 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 22:00:20.859016 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 22:00:20.859016 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 22:00:20.859016 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 22:00:20.859016 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 22:00:20.859016 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 22:00:20.859016 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 22:00:20.859016 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 22:00:20.859016 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 22:00:20.859016 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 22:00:20.859016 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 13 22:00:20.859016 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 13 22:00:20.859016 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 13 22:00:20.859016 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 13 22:00:20.860241 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 22:00:20.860241 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 22:00:20.860241 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 13 22:00:20.860241 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 22:00:20.860241 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 13 22:00:20.860241 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 22:00:20.860241 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 22:00:20.860241 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 22:00:20.860241 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 22:00:20.860241 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 22:00:20.860241 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 22:00:20.860241 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application with Lock( Jul 13 22:00:20.860241 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 22:00:20.860241 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._enter() Jul 13 22:00:20.860241 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 22:00:20.860241 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 22:00:20.860241 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 22:00:20.860241 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.creator() Jul 13 22:00:20.861508 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 22:00:20.861508 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application created_value = creator( Jul 13 22:00:20.861508 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 13 22:00:20.861508 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 13 22:00:20.861508 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 13 22:00:20.861508 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 13 22:00:20.861508 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 13 22:00:20.861508 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 13 22:00:20.861508 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 13 22:00:20.861508 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Jul 13 22:00:20.861508 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 522/1038] 10.222.0.52 () {56 vars in 1093 bytes} [Wed Jul 13 22:00:20 2022] GET /identity/v3/users/nova => generated 82 bytes in 183 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 22:00:20.867418 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed36a4e1-e8a6-4b4c-afa2-aad076b3136d None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:00:20.899759 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:00:20.972467 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed36a4e1-e8a6-4b4c-afa2-aad076b3136d None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:00:21.007128 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50cd659-882a-4130-9a96-4da83147a343 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:00:21.007903 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50cd659-882a-4130-9a96-4da83147a343 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:00:21.008682 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50cd659-882a-4130-9a96-4da83147a343 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:00:21.008962 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b50cd659-882a-4130-9a96-4da83147a343 None admin] PATH_INFO: `/v3/users` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:00:21.009814 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b50cd659-882a-4130-9a96-4da83147a343 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 22:00:21.010122 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b50cd659-882a-4130-9a96-4da83147a343 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:00:21.011586 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b50cd659-882a-4130-9a96-4da83147a343 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:00:21.029611 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 517/1039] 10.222.0.52 () {56 vars in 1133 bytes} [Wed Jul 13 22:00:20 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 22:00:21.035217 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b224d8-7516-4e06-84d2-c162f5a34cb1 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:00:21.141891 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6b224d8-7516-4e06-84d2-c162f5a34cb1 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:00:21.157163 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:00:21.178124 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1b9812-e4a7-447c-a456-d623a210b99b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:00:21.178806 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1b9812-e4a7-447c-a456-d623a210b99b None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:00:21.179087 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1b9812-e4a7-447c-a456-d623a210b99b None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:00:21.179362 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d1b9812-e4a7-447c-a456-d623a210b99b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:00:21.249652 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d1b9812-e4a7-447c-a456-d623a210b99b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:00:21.251193 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d1b9812-e4a7-447c-a456-d623a210b99b None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:00:21.364327 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 523/1040] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Jul 13 22:00:21 2022] GET /identity/v3/auth/tokens => generated 1959 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 13 22:00:21.370063 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50cd659-882a-4130-9a96-4da83147a343 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:00:21.466430 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:00:21.477354 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b50cd659-882a-4130-9a96-4da83147a343 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:00:21.512525 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3cfd306-487f-42c4-a72e-8f727b65cb35 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:00:21.513324 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cfd306-487f-42c4-a72e-8f727b65cb35 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:00:21.513609 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cfd306-487f-42c4-a72e-8f727b65cb35 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:00:21.513861 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3cfd306-487f-42c4-a72e-8f727b65cb35 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:00:21.524085 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3cfd306-487f-42c4-a72e-8f727b65cb35 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:00:21.525438 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a3cfd306-487f-42c4-a72e-8f727b65cb35 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:00:21.543495 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application [None req-a3cfd306-487f-42c4-a72e-8f727b65cb35 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 22:00:21.543495 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 22:00:21.543495 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 22:00:21.543495 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 22:00:21.543495 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 22:00:21.543495 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 22:00:21.543495 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 22:00:21.543495 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 22:00:21.543495 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 22:00:21.543495 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 22:00:21.543495 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 22:00:21.543495 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 22:00:21.543495 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 22:00:21.543495 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 22:00:21.543495 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 22:00:21.544463 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 22:00:21.544463 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 22:00:21.544463 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 22:00:21.544463 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 22:00:21.544463 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 22:00:21.544463 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 22:00:21.544463 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 22:00:21.544463 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 22:00:21.544463 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application with Lock( Jul 13 22:00:21.544463 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 22:00:21.544463 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._enter() Jul 13 22:00:21.544463 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 22:00:21.544463 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 22:00:21.544463 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 22:00:21.544463 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.creator() Jul 13 22:00:21.544463 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 22:00:21.544463 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application created_value = creator( Jul 13 22:00:21.545871 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 22:00:21.545871 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 22:00:21.545871 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 22:00:21.545871 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Jul 13 22:00:21.545871 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 518/1041] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Jul 13 22:00:21 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 22:00:21.554380 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1b9812-e4a7-447c-a456-d623a210b99b None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:00:21.571060 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:00:21.681642 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d1b9812-e4a7-447c-a456-d623a210b99b None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:00:21.721496 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51f5379-d761-4336-9578-360049594aa5 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:00:21.722078 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51f5379-d761-4336-9578-360049594aa5 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:00:21.722470 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51f5379-d761-4336-9578-360049594aa5 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:00:21.722825 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c51f5379-d761-4336-9578-360049594aa5 None admin] PATH_INFO: `/v3/domains` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:00:21.723557 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c51f5379-d761-4336-9578-360049594aa5 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 22:00:21.723860 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c51f5379-d761-4336-9578-360049594aa5 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:00:21.725246 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c51f5379-d761-4336-9578-360049594aa5 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:00:21.745936 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 524/1042] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Jul 13 22:00:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 190 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 22:00:21.747375 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3cfd306-487f-42c4-a72e-8f727b65cb35 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:00:21.882776 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a3cfd306-487f-42c4-a72e-8f727b65cb35 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:00:21.927686 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f5922e-a011-4cc4-b311-6e2b467db2cb None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:00:21.928698 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f5922e-a011-4cc4-b311-6e2b467db2cb None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:00:21.929206 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f5922e-a011-4cc4-b311-6e2b467db2cb None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:00:21.929701 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d8f5922e-a011-4cc4-b311-6e2b467db2cb None admin] PATH_INFO: `/v3/projects/service` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:00:21.937788 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f5922e-a011-4cc4-b311-6e2b467db2cb None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:00:21.939263 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d8f5922e-a011-4cc4-b311-6e2b467db2cb None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:00:21.955107 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application [None req-d8f5922e-a011-4cc4-b311-6e2b467db2cb None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 13 22:00:21.955107 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 22:00:21.955107 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 22:00:21.955107 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 22:00:21.955107 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 22:00:21.955107 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 22:00:21.955107 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 22:00:21.955107 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 22:00:21.955107 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 22:00:21.955107 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 22:00:21.955107 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 22:00:21.955107 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 22:00:21.955107 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 13 22:00:21.955107 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 13 22:00:21.955107 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 13 22:00:21.956081 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 13 22:00:21.956081 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 22:00:21.956081 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 22:00:21.956081 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 22:00:21.956081 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 22:00:21.956081 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 22:00:21.956081 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 22:00:21.956081 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 22:00:21.956081 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application with Lock( Jul 13 22:00:21.956081 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 22:00:21.956081 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._enter() Jul 13 22:00:21.956081 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 22:00:21.956081 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 22:00:21.956081 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 22:00:21.956081 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.creator() Jul 13 22:00:21.956081 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 22:00:21.956081 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application created_value = creator( Jul 13 22:00:21.956992 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 13 22:00:21.956992 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 13 22:00:21.956992 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 13 22:00:21.956992 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 13 22:00:21.956992 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 13 22:00:21.956992 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 13 22:00:21.956992 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 13 22:00:21.956992 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Jul 13 22:00:21.962815 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 519/1043] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Jul 13 22:00:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 22:00:21.967585 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51f5379-d761-4336-9578-360049594aa5 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:00:22.063853 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:00:22.105625 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c51f5379-d761-4336-9578-360049594aa5 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:00:22.150779 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88ad9059-1f97-41e5-a16d-bcc648446016 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:00:22.151767 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ad9059-1f97-41e5-a16d-bcc648446016 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:00:22.152217 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ad9059-1f97-41e5-a16d-bcc648446016 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:00:22.152646 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88ad9059-1f97-41e5-a16d-bcc648446016 None admin] PATH_INFO: `/v3/projects` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:00:22.153655 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ad9059-1f97-41e5-a16d-bcc648446016 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 22:00:22.153964 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ad9059-1f97-41e5-a16d-bcc648446016 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:00:22.155462 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88ad9059-1f97-41e5-a16d-bcc648446016 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:00:22.178571 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 525/1044] 10.222.0.52 () {56 vars in 1148 bytes} [Wed Jul 13 22:00:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 205 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 22:00:22.182418 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f5922e-a011-4cc4-b311-6e2b467db2cb None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:00:22.191176 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:00:22.200776 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28391370-1b3c-47b0-9eb4-092c640f1df2 None None] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:00:22.201089 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28391370-1b3c-47b0-9eb4-092c640f1df2 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:00:22.201546 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28391370-1b3c-47b0-9eb4-092c640f1df2 None None] PATH_INFO: `/` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:00:22.203439 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 526/1045] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jul 13 22:00:22 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 13 22:00:22.208566 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d90f815-17fe-4edb-a81e-3104c76635b0 None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:00:22.208843 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d90f815-17fe-4edb-a81e-3104c76635b0 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:00:22.209102 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d90f815-17fe-4edb-a81e-3104c76635b0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:00:22.226318 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-9d90f815-17fe-4edb-a81e-3104c76635b0 None None] MFA Rules not processed for user `cc99c93da5074cf5a48bc2aa9a92c5e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 22:00:22.227583 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-9d90f815-17fe-4edb-a81e-3104c76635b0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:00:22.307050 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 527/1046] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jul 13 22:00:22 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 13 22:00:22.324630 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d8f5922e-a011-4cc4-b311-6e2b467db2cb None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:00:22.332199 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d90f815-17fe-4edb-a81e-3104c76635b0 None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:00:22.366972 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214a29dd-7b8f-4057-834d-71c7dd089d37 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:00:22.367961 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214a29dd-7b8f-4057-834d-71c7dd089d37 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:00:22.368301 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214a29dd-7b8f-4057-834d-71c7dd089d37 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:00:22.368818 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-214a29dd-7b8f-4057-834d-71c7dd089d37 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:00:22.369814 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-214a29dd-7b8f-4057-834d-71c7dd089d37 None admin] RBAC: Adding query filter params (role.id=3987457f5fe64b568601d117c2c0ba9d, scope.project.id=417e3e2f1333481795a089bef2e24bea, user.id=f84281091687464990ffbca590b9343b) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 22:00:22.370122 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-214a29dd-7b8f-4057-834d-71c7dd089d37 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:00:22.371603 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-214a29dd-7b8f-4057-834d-71c7dd089d37 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:00:22.377954 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 520/1047] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Jul 13 22:00:22 2022] GET /identity/v3/role_assignments?user.id=f84281091687464990ffbca590b9343b&scope.project.id=417e3e2f1333481795a089bef2e24bea&role.id=3987457f5fe64b568601d117c2c0ba9d => generated 259 bytes in 197 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 22:00:22.419720 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:00:22.476855 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d90f815-17fe-4edb-a81e-3104c76635b0 None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:00:22.527701 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d715778c-1c50-48fa-bb23-320cce5ca91f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:00:22.528536 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d715778c-1c50-48fa-bb23-320cce5ca91f None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:00:22.528782 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d715778c-1c50-48fa-bb23-320cce5ca91f None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:00:22.529094 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d715778c-1c50-48fa-bb23-320cce5ca91f None admin] PATH_INFO: `/v3/services/image` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:00:22.529757 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d715778c-1c50-48fa-bb23-320cce5ca91f None admin] RBAC: Authorizing `identity:get_service(service_id=image)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:00:22.531226 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d715778c-1c50-48fa-bb23-320cce5ca91f None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:00:22.547385 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application [None req-d715778c-1c50-48fa-bb23-320cce5ca91f None admin] Could not find service: image.: keystone.exception.ServiceNotFound: Could not find service: image. Jul 13 22:00:22.547385 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 22:00:22.547385 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 13 22:00:22.547385 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 13 22:00:22.547385 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 13 22:00:22.547385 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 13 22:00:22.547385 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 13 22:00:22.547385 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 13 22:00:22.547385 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 13 22:00:22.547385 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Jul 13 22:00:22.547385 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 13 22:00:22.547385 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Jul 13 22:00:22.547385 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 22:00:22.547385 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 22:00:22.547385 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 22:00:22.547385 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 22:00:22.547385 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 22:00:22.548504 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 22:00:22.548504 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 22:00:22.548504 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 22:00:22.548504 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 22:00:22.548504 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 22:00:22.548504 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 22:00:22.548504 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 13 22:00:22.548504 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 13 22:00:22.548504 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 13 22:00:22.548504 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 13 22:00:22.548504 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 22:00:22.548504 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 22:00:22.548504 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 22:00:22.548504 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 22:00:22.548504 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 22:00:22.548504 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 22:00:22.549365 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 22:00:22.549365 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application with Lock( Jul 13 22:00:22.549365 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 22:00:22.549365 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._enter() Jul 13 22:00:22.549365 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 22:00:22.549365 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 22:00:22.549365 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 22:00:22.549365 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.creator() Jul 13 22:00:22.549365 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 22:00:22.549365 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application created_value = creator( Jul 13 22:00:22.549365 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 13 22:00:22.549365 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 13 22:00:22.549365 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: image. Jul 13 22:00:22.549365 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Jul 13 22:00:22.550911 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 528/1048] 10.222.0.52 () {56 vars in 1105 bytes} [Wed Jul 13 22:00:22 2022] GET /identity/v3/services/image => generated 86 bytes in 219 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 22:00:22.555979 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214a29dd-7b8f-4057-834d-71c7dd089d37 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:00:22.624129 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:00:22.661569 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-214a29dd-7b8f-4057-834d-71c7dd089d37 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:00:22.695113 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc29c11a-be7b-464d-b4f8-f1c6a465c4ce None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:00:22.695928 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc29c11a-be7b-464d-b4f8-f1c6a465c4ce None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:00:22.696266 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc29c11a-be7b-464d-b4f8-f1c6a465c4ce None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:00:22.696574 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc29c11a-be7b-464d-b4f8-f1c6a465c4ce None admin] PATH_INFO: `/v3/services` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:00:22.697173 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc29c11a-be7b-464d-b4f8-f1c6a465c4ce None admin] RBAC: Adding query filter params (name=image) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 22:00:22.697573 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc29c11a-be7b-464d-b4f8-f1c6a465c4ce None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:00:22.699340 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc29c11a-be7b-464d-b4f8-f1c6a465c4ce None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:00:22.705022 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 521/1049] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Jul 13 22:00:22 2022] GET /identity/v3/services?name=image => generated 122 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 22:00:22.710876 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d715778c-1c50-48fa-bb23-320cce5ca91f None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:00:22.811778 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d715778c-1c50-48fa-bb23-320cce5ca91f None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:00:22.835310 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6f7886-1e01-4655-b3de-941230f62ba4 None None] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:00:22.835599 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6f7886-1e01-4655-b3de-941230f62ba4 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:00:22.835958 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b6f7886-1e01-4655-b3de-941230f62ba4 None None] PATH_INFO: `/` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:00:22.837502 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 522/1050] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jul 13 22:00:22 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 13 22:00:22.839063 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:00:22.843231 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f46846-5bd0-4512-ac6b-5d402539983c None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:00:22.843607 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f46846-5bd0-4512-ac6b-5d402539983c None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:00:22.844099 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14f46846-5bd0-4512-ac6b-5d402539983c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:00:22.847960 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53832edb-1540-41db-89d2-fca2e49a9dee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:00:22.848630 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53832edb-1540-41db-89d2-fca2e49a9dee None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:00:22.848906 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53832edb-1540-41db-89d2-fca2e49a9dee None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:00:22.849176 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53832edb-1540-41db-89d2-fca2e49a9dee None admin] PATH_INFO: `/v3/services` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:00:22.849884 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53832edb-1540-41db-89d2-fca2e49a9dee None admin] RBAC: Adding query filter params (type=image) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 22:00:22.850187 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53832edb-1540-41db-89d2-fca2e49a9dee None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:00:22.851644 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53832edb-1540-41db-89d2-fca2e49a9dee None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:00:22.862095 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 529/1051] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Jul 13 22:00:22 2022] GET /identity/v3/services?type=image => generated 350 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 22:00:22.863369 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53832edb-1540-41db-89d2-fca2e49a9dee None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:00:22.863853 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-14f46846-5bd0-4512-ac6b-5d402539983c None None] MFA Rules not processed for user `cc99c93da5074cf5a48bc2aa9a92c5e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 22:00:22.865017 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-14f46846-5bd0-4512-ac6b-5d402539983c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:00:22.936539 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 523/1052] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jul 13 22:00:22 2022] POST /identity/v3/auth/tokens => generated 1959 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 13 22:00:22.960042 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f46846-5bd0-4512-ac6b-5d402539983c None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:00:22.973893 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53832edb-1540-41db-89d2-fca2e49a9dee None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:00:23.010418 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f274022d-0192-4e56-93f8-047dd1a4fd8d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:00:23.011528 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f274022d-0192-4e56-93f8-047dd1a4fd8d None admin] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:00:23.012020 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f274022d-0192-4e56-93f8-047dd1a4fd8d None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:00:23.012505 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f274022d-0192-4e56-93f8-047dd1a4fd8d None admin] PATH_INFO: `/v3/endpoints` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:00:23.013278 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f274022d-0192-4e56-93f8-047dd1a4fd8d None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:00:23.014687 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f274022d-0192-4e56-93f8-047dd1a4fd8d None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:00:23.024500 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:00:23.037139 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 530/1053] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Jul 13 22:00:22 2022] POST /identity/v3/endpoints => generated 327 bytes in 174 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 13 22:00:23.082748 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14f46846-5bd0-4512-ac6b-5d402539983c None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:00:23.128503 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87d9f0f-f7a9-48b4-b3f3-c648a8638c80 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:00:23.129317 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87d9f0f-f7a9-48b4-b3f3-c648a8638c80 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:00:23.129592 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87d9f0f-f7a9-48b4-b3f3-c648a8638c80 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:00:23.129861 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a87d9f0f-f7a9-48b4-b3f3-c648a8638c80 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:00:23.141487 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a87d9f0f-f7a9-48b4-b3f3-c648a8638c80 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 22:00:23.149650 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application [None req-a87d9f0f-f7a9-48b4-b3f3-c648a8638c80 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 13 22:00:23.149650 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 22:00:23.149650 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 22:00:23.149650 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 22:00:23.149650 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 22:00:23.149650 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 22:00:23.149650 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 22:00:23.149650 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 22:00:23.149650 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 22:00:23.149650 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 22:00:23.149650 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 22:00:23.149650 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 22:00:23.149650 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 22:00:23.149650 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 22:00:23.149650 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 22:00:23.149650 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 22:00:23.156231 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 22:00:23.156231 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 22:00:23.156231 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 22:00:23.156231 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 22:00:23.156231 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 22:00:23.156231 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 22:00:23.156231 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 22:00:23.156231 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 22:00:23.156231 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 22:00:23.156231 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 22:00:23.156231 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 22:00:23.156231 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application with Lock( Jul 13 22:00:23.156231 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 22:00:23.156231 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._enter() Jul 13 22:00:23.156231 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 22:00:23.156231 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 22:00:23.157126 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 22:00:23.157126 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.creator() Jul 13 22:00:23.157126 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 22:00:23.157126 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application created_value = creator( Jul 13 22:00:23.157126 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 22:00:23.157126 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 22:00:23.157126 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 22:00:23.157126 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 22:00:23.157126 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 22:00:23.157126 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 22:00:23.157126 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 13 22:00:23.157126 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Jul 13 22:00:23.157126 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 524/1054] 10.222.0.52 () {56 vars in 1096 bytes} [Wed Jul 13 22:00:22 2022] GET /identity/v3/roles/admin => generated 83 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 13 22:00:23.159476 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f274022d-0192-4e56-93f8-047dd1a4fd8d None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:00:23.218361 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:00:23.274668 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f274022d-0192-4e56-93f8-047dd1a4fd8d None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:00:23.309989 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3731c14e-e5fc-4898-9b99-204cce817ce1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:00:23.310700 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3731c14e-e5fc-4898-9b99-204cce817ce1 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:00:23.310981 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3731c14e-e5fc-4898-9b99-204cce817ce1 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:00:23.311253 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3731c14e-e5fc-4898-9b99-204cce817ce1 None admin] PATH_INFO: `/v3/roles` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:00:23.312003 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3731c14e-e5fc-4898-9b99-204cce817ce1 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 22:00:23.312311 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3731c14e-e5fc-4898-9b99-204cce817ce1 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:00:23.313689 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3731c14e-e5fc-4898-9b99-204cce817ce1 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:00:23.322605 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 531/1055] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Jul 13 22:00:23 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 22:00:23.328801 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87d9f0f-f7a9-48b4-b3f3-c648a8638c80 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:00:23.432925 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a87d9f0f-f7a9-48b4-b3f3-c648a8638c80 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:00:23.450507 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:00:23.470323 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d36853-db92-4476-97a6-21eb3d25e341 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:00:23.471176 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d36853-db92-4476-97a6-21eb3d25e341 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:00:23.471460 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d36853-db92-4476-97a6-21eb3d25e341 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:00:23.471737 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7d36853-db92-4476-97a6-21eb3d25e341 None admin] PATH_INFO: `/v3/users/93d1c557bab54a69beb7295127463636` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:00:23.486319 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d36853-db92-4476-97a6-21eb3d25e341 None admin] RBAC: Authorizing `identity:get_user(user_id=93d1c557bab54a69beb7295127463636)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:00:23.487820 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7d36853-db92-4476-97a6-21eb3d25e341 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:00:23.489734 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 525/1056] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Jul 13 22:00:23 2022] GET /identity/v3/users/93d1c557bab54a69beb7295127463636 => generated 285 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 22:00:23.495424 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3731c14e-e5fc-4898-9b99-204cce817ce1 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:00:23.611126 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3731c14e-e5fc-4898-9b99-204cce817ce1 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:00:23.621760 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:00:23.648558 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36861c8a-6cf5-4ab7-91d8-8b06c28c7a6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:00:23.649469 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36861c8a-6cf5-4ab7-91d8-8b06c28c7a6b None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:00:23.649901 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36861c8a-6cf5-4ab7-91d8-8b06c28c7a6b None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:00:23.650325 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36861c8a-6cf5-4ab7-91d8-8b06c28c7a6b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:00:23.723468 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36861c8a-6cf5-4ab7-91d8-8b06c28c7a6b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:00:23.725166 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36861c8a-6cf5-4ab7-91d8-8b06c28c7a6b None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:00:23.828456 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 532/1057] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Jul 13 22:00:23 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 13 22:00:23.834022 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d36853-db92-4476-97a6-21eb3d25e341 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:00:23.937457 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7d36853-db92-4476-97a6-21eb3d25e341 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:00:23.972943 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08197264-5a71-4ea8-ad65-119bc24f3e57 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:00:23.973878 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08197264-5a71-4ea8-ad65-119bc24f3e57 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:00:23.974091 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08197264-5a71-4ea8-ad65-119bc24f3e57 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:00:23.974352 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08197264-5a71-4ea8-ad65-119bc24f3e57 None admin] PATH_INFO: `/v3/projects/3d8921c7315943f4a11901f37140ba1e` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:00:23.975613 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08197264-5a71-4ea8-ad65-119bc24f3e57 None admin] RBAC: Authorizing `identity:get_project(project_id=3d8921c7315943f4a11901f37140ba1e)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:00:23.977108 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08197264-5a71-4ea8-ad65-119bc24f3e57 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:00:23.979109 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 526/1058] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Jul 13 22:00:23 2022] GET /identity/v3/projects/3d8921c7315943f4a11901f37140ba1e => generated 311 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 22:00:23.985109 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36861c8a-6cf5-4ab7-91d8-8b06c28c7a6b None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:00:23.998001 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:00:24.090419 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36861c8a-6cf5-4ab7-91d8-8b06c28c7a6b None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:00:24.124701 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b3d0c66-35c0-4457-8267-0ecd0b21e90e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:00:24.125368 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3d0c66-35c0-4457-8267-0ecd0b21e90e None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:00:24.125652 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3d0c66-35c0-4457-8267-0ecd0b21e90e None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:00:24.125929 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b3d0c66-35c0-4457-8267-0ecd0b21e90e None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:00:24.126792 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3d0c66-35c0-4457-8267-0ecd0b21e90e None admin] RBAC: Adding query filter params (role.id=44efba2579f342df8a0789c59d163ec3, scope.project.id=3d8921c7315943f4a11901f37140ba1e, user.id=93d1c557bab54a69beb7295127463636) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 22:00:24.127104 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3d0c66-35c0-4457-8267-0ecd0b21e90e None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:00:24.128538 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b3d0c66-35c0-4457-8267-0ecd0b21e90e None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:00:24.138590 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 533/1059] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Jul 13 22:00:23 2022] GET /identity/v3/role_assignments?user.id=93d1c557bab54a69beb7295127463636&scope.project.id=3d8921c7315943f4a11901f37140ba1e&role.id=44efba2579f342df8a0789c59d163ec3 => generated 259 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 22:00:24.460229 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0c846f-b054-4dd7-a3f9-c6a836198978 None None] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:00:24.460914 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0c846f-b054-4dd7-a3f9-c6a836198978 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:00:24.460914 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f0c846f-b054-4dd7-a3f9-c6a836198978 None None] PATH_INFO: `/` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:00:24.463050 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 527/1060] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jul 13 22:00:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 22:00:24.468267 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6e3f95-3770-4079-8c9f-490f68d3a2a3 None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:00:24.468572 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6e3f95-3770-4079-8c9f-490f68d3a2a3 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:00:24.468812 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b6e3f95-3770-4079-8c9f-490f68d3a2a3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:00:24.486569 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-6b6e3f95-3770-4079-8c9f-490f68d3a2a3 None None] MFA Rules not processed for user `cc99c93da5074cf5a48bc2aa9a92c5e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 22:00:24.487742 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-6b6e3f95-3770-4079-8c9f-490f68d3a2a3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:00:24.570945 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 534/1061] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jul 13 22:00:24 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 13 22:00:24.594955 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0c846f-b054-4dd7-a3f9-c6a836198978 None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:00:24.642767 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:00:24.702205 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f0c846f-b054-4dd7-a3f9-c6a836198978 None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:00:24.736238 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64fdd62b-5d72-4b0f-a083-efb0c317cc43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:00:24.736978 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fdd62b-5d72-4b0f-a083-efb0c317cc43 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:00:24.737260 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fdd62b-5d72-4b0f-a083-efb0c317cc43 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:00:24.737533 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64fdd62b-5d72-4b0f-a083-efb0c317cc43 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:00:24.747969 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64fdd62b-5d72-4b0f-a083-efb0c317cc43 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 22:00:24.756199 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application [None req-64fdd62b-5d72-4b0f-a083-efb0c317cc43 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 13 22:00:24.756199 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 22:00:24.756199 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 22:00:24.756199 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 22:00:24.756199 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 22:00:24.756199 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 22:00:24.756199 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 22:00:24.756199 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 22:00:24.756199 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 22:00:24.756199 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 22:00:24.756199 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 22:00:24.756199 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 22:00:24.756199 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 22:00:24.756199 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 22:00:24.756199 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 22:00:24.756199 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 22:00:24.757148 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 22:00:24.757148 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 22:00:24.757148 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 22:00:24.757148 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 22:00:24.757148 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 22:00:24.757148 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 22:00:24.757148 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 22:00:24.757148 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 22:00:24.757148 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 22:00:24.757148 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 22:00:24.757148 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 22:00:24.757148 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application with Lock( Jul 13 22:00:24.757148 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 22:00:24.757148 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._enter() Jul 13 22:00:24.757148 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 22:00:24.757148 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 22:00:24.758121 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 22:00:24.758121 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.creator() Jul 13 22:00:24.758121 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 22:00:24.758121 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application created_value = creator( Jul 13 22:00:24.758121 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 22:00:24.758121 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 22:00:24.758121 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 22:00:24.758121 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 22:00:24.758121 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 22:00:24.758121 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 22:00:24.758121 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 13 22:00:24.758121 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Jul 13 22:00:24.759528 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 528/1062] 10.222.0.52 () {56 vars in 1120 bytes} [Wed Jul 13 22:00:24 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 22:00:24.764257 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6e3f95-3770-4079-8c9f-490f68d3a2a3 None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:00:24.767555 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:00:24.869781 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b6e3f95-3770-4079-8c9f-490f68d3a2a3 None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:00:24.903883 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95cc485c-a8ac-4932-9529-98fc7fdbd8cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:00:24.904561 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cc485c-a8ac-4932-9529-98fc7fdbd8cc None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:00:24.904840 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cc485c-a8ac-4932-9529-98fc7fdbd8cc None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:00:24.905113 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95cc485c-a8ac-4932-9529-98fc7fdbd8cc None admin] PATH_INFO: `/v3/roles` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:00:24.905868 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95cc485c-a8ac-4932-9529-98fc7fdbd8cc None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 22:00:24.906178 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95cc485c-a8ac-4932-9529-98fc7fdbd8cc None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:00:24.907598 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95cc485c-a8ac-4932-9529-98fc7fdbd8cc None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:00:24.915919 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 535/1063] 10.222.0.52 () {56 vars in 1115 bytes} [Wed Jul 13 22:00:24 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 22:00:24.921498 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64fdd62b-5d72-4b0f-a083-efb0c317cc43 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:00:25.021271 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64fdd62b-5d72-4b0f-a083-efb0c317cc43 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:00:25.022742 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:00:25.056779 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528dbf21-c4ff-4096-bde4-78b7a926e937 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:00:25.057585 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528dbf21-c4ff-4096-bde4-78b7a926e937 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:00:25.057862 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528dbf21-c4ff-4096-bde4-78b7a926e937 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:00:25.058136 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528dbf21-c4ff-4096-bde4-78b7a926e937 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:00:25.069018 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528dbf21-c4ff-4096-bde4-78b7a926e937 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:00:25.070665 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-528dbf21-c4ff-4096-bde4-78b7a926e937 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:00:25.074790 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014b71a4-9a36-4075-874f-043074e11a0d None None] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:00:25.075073 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014b71a4-9a36-4075-874f-043074e11a0d None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:00:25.075347 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014b71a4-9a36-4075-874f-043074e11a0d None None] PATH_INFO: `/` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:00:25.076855 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 536/1064] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jul 13 22:00:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 13 22:00:25.081711 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4ac548-9ad4-4f81-be0a-a353b78dbbf1 None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:00:25.082060 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4ac548-9ad4-4f81-be0a-a353b78dbbf1 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:00:25.082395 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b4ac548-9ad4-4f81-be0a-a353b78dbbf1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:00:25.092118 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application [None req-528dbf21-c4ff-4096-bde4-78b7a926e937 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 22:00:25.092118 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 22:00:25.092118 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 22:00:25.092118 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 22:00:25.092118 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 22:00:25.092118 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 22:00:25.092118 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 22:00:25.092118 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 22:00:25.092118 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 22:00:25.092118 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 22:00:25.092118 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 22:00:25.092118 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 22:00:25.092118 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 22:00:25.092118 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 22:00:25.092118 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 22:00:25.093071 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 22:00:25.093071 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 22:00:25.093071 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 22:00:25.093071 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 22:00:25.093071 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 22:00:25.093071 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 22:00:25.093071 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 22:00:25.093071 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 22:00:25.093071 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application with Lock( Jul 13 22:00:25.093071 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 22:00:25.093071 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._enter() Jul 13 22:00:25.093071 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 22:00:25.093071 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 22:00:25.093071 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 22:00:25.093071 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.creator() Jul 13 22:00:25.093071 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 22:00:25.093071 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application created_value = creator( Jul 13 22:00:25.093996 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 22:00:25.093996 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 22:00:25.093996 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 22:00:25.093996 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Jul 13 22:00:25.093996 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 529/1065] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Jul 13 22:00:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 22:00:25.099000 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528dbf21-c4ff-4096-bde4-78b7a926e937 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:00:25.104986 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-0b4ac548-9ad4-4f81-be0a-a353b78dbbf1 None None] MFA Rules not processed for user `cc99c93da5074cf5a48bc2aa9a92c5e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 22:00:25.106168 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-0b4ac548-9ad4-4f81-be0a-a353b78dbbf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:00:25.167048 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:00:25.177150 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 537/1066] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jul 13 22:00:25 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 13 22:00:25.200136 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4ac548-9ad4-4f81-be0a-a353b78dbbf1 None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:00:25.207719 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-528dbf21-c4ff-4096-bde4-78b7a926e937 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:00:25.245261 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf07a9b-763e-4cbf-a054-a3d8166fb0e9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:00:25.245937 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf07a9b-763e-4cbf-a054-a3d8166fb0e9 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:00:25.246219 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf07a9b-763e-4cbf-a054-a3d8166fb0e9 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:00:25.246504 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9bf07a9b-763e-4cbf-a054-a3d8166fb0e9 None admin] PATH_INFO: `/v3/domains` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:00:25.247243 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf07a9b-763e-4cbf-a054-a3d8166fb0e9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 22:00:25.247556 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf07a9b-763e-4cbf-a054-a3d8166fb0e9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:00:25.248925 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9bf07a9b-763e-4cbf-a054-a3d8166fb0e9 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:00:25.262537 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 530/1067] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Jul 13 22:00:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 22:00:25.268189 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf07a9b-763e-4cbf-a054-a3d8166fb0e9 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:00:25.289013 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:00:25.312841 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b4ac548-9ad4-4f81-be0a-a353b78dbbf1 None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:00:25.348242 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972e423f-75a3-4336-a413-93fbe1197b8a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:00:25.348903 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972e423f-75a3-4336-a413-93fbe1197b8a None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:00:25.349185 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972e423f-75a3-4336-a413-93fbe1197b8a None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:00:25.349461 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-972e423f-75a3-4336-a413-93fbe1197b8a None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:00:25.381659 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9bf07a9b-763e-4cbf-a054-a3d8166fb0e9 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:00:25.416761 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1025a7cb-d983-4847-835c-890151434cec None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:00:25.417557 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1025a7cb-d983-4847-835c-890151434cec None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:00:25.417835 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1025a7cb-d983-4847-835c-890151434cec None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:00:25.418110 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1025a7cb-d983-4847-835c-890151434cec None admin] PATH_INFO: `/v3/users/nova` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:00:25.418675 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:00:25.425797 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1025a7cb-d983-4847-835c-890151434cec None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:00:25.426618 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972e423f-75a3-4336-a413-93fbe1197b8a None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:00:25.427454 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1025a7cb-d983-4847-835c-890151434cec None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:00:25.428509 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-972e423f-75a3-4336-a413-93fbe1197b8a None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:00:25.442408 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application [None req-1025a7cb-d983-4847-835c-890151434cec None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 13 22:00:25.442408 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 22:00:25.442408 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 22:00:25.442408 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 22:00:25.442408 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 22:00:25.442408 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 22:00:25.442408 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 22:00:25.442408 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 22:00:25.442408 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 22:00:25.442408 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 22:00:25.442408 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 22:00:25.442408 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 22:00:25.442408 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 13 22:00:25.442408 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 13 22:00:25.442408 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 13 22:00:25.442408 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 13 22:00:25.443623 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 22:00:25.443623 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 22:00:25.443623 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 13 22:00:25.443623 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 22:00:25.443623 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 13 22:00:25.443623 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 22:00:25.443623 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 22:00:25.443623 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 22:00:25.443623 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 22:00:25.443623 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 22:00:25.443623 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 22:00:25.443623 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application with Lock( Jul 13 22:00:25.443623 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 22:00:25.443623 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._enter() Jul 13 22:00:25.443623 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 22:00:25.443623 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 22:00:25.443623 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 22:00:25.443623 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.creator() Jul 13 22:00:25.444580 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 22:00:25.444580 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application created_value = creator( Jul 13 22:00:25.444580 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 13 22:00:25.444580 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 13 22:00:25.444580 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 13 22:00:25.444580 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 13 22:00:25.444580 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 13 22:00:25.444580 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 13 22:00:25.444580 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 13 22:00:25.444580 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Jul 13 22:00:25.444580 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 531/1068] 10.222.0.52 () {56 vars in 1093 bytes} [Wed Jul 13 22:00:25 2022] GET /identity/v3/users/nova => generated 82 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 13 22:00:25.449917 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1025a7cb-d983-4847-835c-890151434cec None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:00:25.526944 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:00:25.536949 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 538/1069] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Jul 13 22:00:25 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 13 22:00:25.543023 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972e423f-75a3-4336-a413-93fbe1197b8a None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:00:25.561013 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1025a7cb-d983-4847-835c-890151434cec None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:00:25.600684 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a8a684-38cc-4022-bd08-cc7a7ed08e73 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:00:25.601377 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a8a684-38cc-4022-bd08-cc7a7ed08e73 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:00:25.601767 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a8a684-38cc-4022-bd08-cc7a7ed08e73 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:00:25.602249 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62a8a684-38cc-4022-bd08-cc7a7ed08e73 None admin] PATH_INFO: `/v3/users` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:00:25.603183 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a8a684-38cc-4022-bd08-cc7a7ed08e73 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 22:00:25.603492 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a8a684-38cc-4022-bd08-cc7a7ed08e73 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:00:25.604931 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62a8a684-38cc-4022-bd08-cc7a7ed08e73 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:00:25.622424 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 532/1070] 10.222.0.52 () {56 vars in 1133 bytes} [Wed Jul 13 22:00:25 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 173 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 22:00:25.628172 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a8a684-38cc-4022-bd08-cc7a7ed08e73 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:00:25.648363 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:00:25.656564 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-972e423f-75a3-4336-a413-93fbe1197b8a None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:00:25.700120 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b54113-4c40-4aff-b2b6-accc15990b6b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:00:25.701191 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b54113-4c40-4aff-b2b6-accc15990b6b None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:00:25.701633 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b54113-4c40-4aff-b2b6-accc15990b6b None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:00:25.701908 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4b54113-4c40-4aff-b2b6-accc15990b6b None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:00:25.712914 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b54113-4c40-4aff-b2b6-accc15990b6b None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:00:25.714647 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d4b54113-4c40-4aff-b2b6-accc15990b6b None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:00:25.732937 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application [None req-d4b54113-4c40-4aff-b2b6-accc15990b6b None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 22:00:25.732937 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 22:00:25.732937 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 22:00:25.732937 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 22:00:25.732937 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 22:00:25.732937 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 22:00:25.732937 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 22:00:25.732937 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 22:00:25.732937 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 22:00:25.732937 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 22:00:25.732937 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 22:00:25.732937 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 22:00:25.732937 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 22:00:25.732937 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 22:00:25.732937 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 22:00:25.733917 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 22:00:25.733917 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 22:00:25.733917 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 22:00:25.733917 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 22:00:25.733917 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 22:00:25.733917 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 22:00:25.733917 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 22:00:25.733917 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 22:00:25.733917 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application with Lock( Jul 13 22:00:25.733917 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 22:00:25.733917 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._enter() Jul 13 22:00:25.733917 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 22:00:25.733917 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 22:00:25.733917 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 22:00:25.733917 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.creator() Jul 13 22:00:25.733917 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 22:00:25.733917 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application created_value = creator( Jul 13 22:00:25.734878 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 22:00:25.734878 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 22:00:25.734878 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 22:00:25.734878 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Jul 13 22:00:25.735114 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62a8a684-38cc-4022-bd08-cc7a7ed08e73 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:00:25.735191 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 539/1071] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Jul 13 22:00:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 22:00:25.740020 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b54113-4c40-4aff-b2b6-accc15990b6b None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:00:25.770694 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:00:25.777382 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806a6a17-024c-4900-bde9-4d60c630fe91 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:00:25.778189 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806a6a17-024c-4900-bde9-4d60c630fe91 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:00:25.778464 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806a6a17-024c-4900-bde9-4d60c630fe91 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:00:25.778767 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-806a6a17-024c-4900-bde9-4d60c630fe91 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:00:25.790454 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-806a6a17-024c-4900-bde9-4d60c630fe91 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:00:25.791940 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-806a6a17-024c-4900-bde9-4d60c630fe91 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:00:25.810400 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application [None req-806a6a17-024c-4900-bde9-4d60c630fe91 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 22:00:25.810400 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 22:00:25.810400 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 22:00:25.810400 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 22:00:25.810400 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 22:00:25.810400 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 22:00:25.810400 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 22:00:25.810400 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 22:00:25.810400 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 22:00:25.810400 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 22:00:25.810400 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 22:00:25.810400 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 22:00:25.810400 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 22:00:25.810400 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 22:00:25.810400 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 22:00:25.811361 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 22:00:25.811361 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 22:00:25.811361 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 22:00:25.811361 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 22:00:25.811361 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 22:00:25.811361 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 22:00:25.811361 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 22:00:25.811361 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 22:00:25.811361 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application with Lock( Jul 13 22:00:25.811361 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 22:00:25.811361 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._enter() Jul 13 22:00:25.811361 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 22:00:25.811361 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 22:00:25.811361 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 22:00:25.811361 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.creator() Jul 13 22:00:25.811361 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 22:00:25.811361 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application created_value = creator( Jul 13 22:00:25.812423 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 22:00:25.812423 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 22:00:25.812423 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 22:00:25.812423 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Jul 13 22:00:25.812423 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 533/1072] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Jul 13 22:00:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 13 22:00:25.817281 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806a6a17-024c-4900-bde9-4d60c630fe91 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:00:25.845468 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4b54113-4c40-4aff-b2b6-accc15990b6b None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:00:25.888441 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e9dec6-8aed-47a4-b7ff-77c8517da411 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:00:25.889533 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e9dec6-8aed-47a4-b7ff-77c8517da411 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:00:25.889830 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e9dec6-8aed-47a4-b7ff-77c8517da411 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:00:25.890238 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32e9dec6-8aed-47a4-b7ff-77c8517da411 None admin] PATH_INFO: `/v3/domains` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:00:25.891112 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e9dec6-8aed-47a4-b7ff-77c8517da411 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 22:00:25.891561 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e9dec6-8aed-47a4-b7ff-77c8517da411 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:00:25.893557 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32e9dec6-8aed-47a4-b7ff-77c8517da411 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:00:25.905888 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 540/1073] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Jul 13 22:00:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 22:00:25.938577 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-806a6a17-024c-4900-bde9-4d60c630fe91 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:00:25.980470 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dad75c61-85a7-4dce-bd42-b85798838f25 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:00:25.981155 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad75c61-85a7-4dce-bd42-b85798838f25 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:00:25.981438 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad75c61-85a7-4dce-bd42-b85798838f25 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:00:25.981710 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad75c61-85a7-4dce-bd42-b85798838f25 None admin] PATH_INFO: `/v3/domains` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:00:25.982451 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad75c61-85a7-4dce-bd42-b85798838f25 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 22:00:25.982779 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad75c61-85a7-4dce-bd42-b85798838f25 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:00:25.984178 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dad75c61-85a7-4dce-bd42-b85798838f25 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:00:25.998367 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 534/1074] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Jul 13 22:00:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 181 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 22:00:26.004064 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e9dec6-8aed-47a4-b7ff-77c8517da411 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:00:26.016888 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:00:26.119399 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32e9dec6-8aed-47a4-b7ff-77c8517da411 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:00:26.136222 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ce5ddf-7705-4fbf-be1b-4c67b1f88ca5 None None] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:00:26.136546 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ce5ddf-7705-4fbf-be1b-4c67b1f88ca5 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:00:26.136786 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4ce5ddf-7705-4fbf-be1b-4c67b1f88ca5 None None] PATH_INFO: `/` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:00:26.138186 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 535/1075] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jul 13 22:00:26 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 13 22:00:26.143326 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c860c7-4661-4fa1-a8bd-7a502c2d6916 None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:00:26.143608 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c860c7-4661-4fa1-a8bd-7a502c2d6916 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:00:26.143894 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c860c7-4661-4fa1-a8bd-7a502c2d6916 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:00:26.157153 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd50568-0b3c-41c6-a540-1d588ab6873b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:00:26.158074 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd50568-0b3c-41c6-a540-1d588ab6873b None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:00:26.158359 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd50568-0b3c-41c6-a540-1d588ab6873b None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:00:26.158691 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cd50568-0b3c-41c6-a540-1d588ab6873b None admin] PATH_INFO: `/v3/projects/service` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:00:26.165287 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-46c860c7-4661-4fa1-a8bd-7a502c2d6916 None None] MFA Rules not processed for user `cc99c93da5074cf5a48bc2aa9a92c5e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 22:00:26.165770 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd50568-0b3c-41c6-a540-1d588ab6873b None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:00:26.166624 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-46c860c7-4661-4fa1-a8bd-7a502c2d6916 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:00:26.167281 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cd50568-0b3c-41c6-a540-1d588ab6873b None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:00:26.182099 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application [None req-0cd50568-0b3c-41c6-a540-1d588ab6873b None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 13 22:00:26.182099 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 22:00:26.182099 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 22:00:26.182099 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 22:00:26.182099 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 22:00:26.182099 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 22:00:26.182099 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 22:00:26.182099 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 22:00:26.182099 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 22:00:26.182099 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 22:00:26.182099 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 22:00:26.182099 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 22:00:26.182099 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 13 22:00:26.182099 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 13 22:00:26.182099 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 13 22:00:26.183140 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 13 22:00:26.183140 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 22:00:26.183140 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 22:00:26.183140 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 22:00:26.183140 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 22:00:26.183140 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 22:00:26.183140 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 22:00:26.183140 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 22:00:26.183140 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application with Lock( Jul 13 22:00:26.183140 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 22:00:26.183140 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._enter() Jul 13 22:00:26.183140 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 22:00:26.183140 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 22:00:26.183140 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 22:00:26.183140 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.creator() Jul 13 22:00:26.183140 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 22:00:26.183140 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application created_value = creator( Jul 13 22:00:26.184071 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 13 22:00:26.184071 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 13 22:00:26.184071 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 13 22:00:26.184071 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 13 22:00:26.184071 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 13 22:00:26.184071 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 13 22:00:26.184071 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 13 22:00:26.184071 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Jul 13 22:00:26.184071 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 541/1076] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Jul 13 22:00:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 22:00:26.189098 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd50568-0b3c-41c6-a540-1d588ab6873b None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:00:26.214104 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:00:26.250259 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 536/1077] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jul 13 22:00:26 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 13 22:00:26.257077 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:00:26.272803 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c860c7-4661-4fa1-a8bd-7a502c2d6916 None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:00:26.294258 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cd50568-0b3c-41c6-a540-1d588ab6873b None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:00:26.329016 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142c9f55-a81f-4337-9d99-6cb827eef022 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:00:26.329692 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142c9f55-a81f-4337-9d99-6cb827eef022 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:00:26.330003 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142c9f55-a81f-4337-9d99-6cb827eef022 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:00:26.330336 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-142c9f55-a81f-4337-9d99-6cb827eef022 None admin] PATH_INFO: `/v3/projects` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:00:26.331159 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142c9f55-a81f-4337-9d99-6cb827eef022 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 22:00:26.331465 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142c9f55-a81f-4337-9d99-6cb827eef022 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:00:26.332953 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-142c9f55-a81f-4337-9d99-6cb827eef022 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:00:26.345020 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 542/1078] 10.222.0.52 () {56 vars in 1148 bytes} [Wed Jul 13 22:00:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 22:00:26.350950 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142c9f55-a81f-4337-9d99-6cb827eef022 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:00:26.381530 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c860c7-4661-4fa1-a8bd-7a502c2d6916 None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:00:26.415857 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10d0b9e-c175-42bf-b641-3aabe6cb21ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:00:26.416656 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10d0b9e-c175-42bf-b641-3aabe6cb21ed None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:00:26.416934 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10d0b9e-c175-42bf-b641-3aabe6cb21ed None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:00:26.417200 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b10d0b9e-c175-42bf-b641-3aabe6cb21ed None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:00:26.427154 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b10d0b9e-c175-42bf-b641-3aabe6cb21ed None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 22:00:26.434836 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application [None req-b10d0b9e-c175-42bf-b641-3aabe6cb21ed None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 13 22:00:26.434836 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 22:00:26.434836 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 22:00:26.434836 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 22:00:26.434836 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 22:00:26.434836 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 22:00:26.434836 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 22:00:26.434836 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 22:00:26.434836 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 22:00:26.434836 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 22:00:26.434836 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 22:00:26.434836 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 22:00:26.434836 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 22:00:26.434836 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 22:00:26.434836 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 22:00:26.434836 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 22:00:26.435783 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 22:00:26.435783 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 22:00:26.435783 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 22:00:26.435783 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 22:00:26.435783 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 22:00:26.435783 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 22:00:26.435783 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 22:00:26.435783 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 22:00:26.435783 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 22:00:26.435783 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 22:00:26.435783 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 22:00:26.435783 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application with Lock( Jul 13 22:00:26.435783 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 22:00:26.435783 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._enter() Jul 13 22:00:26.435783 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 22:00:26.435783 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 22:00:26.436691 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 22:00:26.436691 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.creator() Jul 13 22:00:26.436691 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 22:00:26.436691 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application created_value = creator( Jul 13 22:00:26.436691 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 22:00:26.436691 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 22:00:26.436691 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 22:00:26.436691 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 22:00:26.436691 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 22:00:26.436691 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 22:00:26.436691 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 13 22:00:26.436691 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Jul 13 22:00:26.436691 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 537/1079] 10.222.0.52 () {56 vars in 1096 bytes} [Wed Jul 13 22:00:26 2022] GET /identity/v3/roles/admin => generated 83 bytes in 164 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 13 22:00:26.443764 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10d0b9e-c175-42bf-b641-3aabe6cb21ed None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:00:26.453018 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-142c9f55-a81f-4337-9d99-6cb827eef022 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:00:26.478507 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:00:26.479745 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:00:26.487331 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fd5c3b-102f-4faf-a542-b3c5d2ea7ae3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:00:26.488230 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fd5c3b-102f-4faf-a542-b3c5d2ea7ae3 None admin] REQUEST_METHOD: `PUT` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:00:26.488503 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fd5c3b-102f-4faf-a542-b3c5d2ea7ae3 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:00:26.488802 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91fd5c3b-102f-4faf-a542-b3c5d2ea7ae3 None admin] PATH_INFO: `/v3/projects/417e3e2f1333481795a089bef2e24bea/users/f84281091687464990ffbca590b9343b/roles/3987457f5fe64b568601d117c2c0ba9d` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:00:26.491062 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91fd5c3b-102f-4faf-a542-b3c5d2ea7ae3 None admin] RBAC: Authorizing `identity:create_grant(project_id=417e3e2f1333481795a089bef2e24bea, user_id=f84281091687464990ffbca590b9343b, role_id=3987457f5fe64b568601d117c2c0ba9d)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:00:26.492534 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91fd5c3b-102f-4faf-a542-b3c5d2ea7ae3 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:00:26.506325 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 543/1080] 10.222.0.52 () {58 vars in 1439 bytes} [Wed Jul 13 22:00:26 2022] PUT /identity/v3/projects/417e3e2f1333481795a089bef2e24bea/users/f84281091687464990ffbca590b9343b/roles/3987457f5fe64b568601d117c2c0ba9d => generated 0 bytes in 156 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 13 22:00:26.554262 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b10d0b9e-c175-42bf-b641-3aabe6cb21ed None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:00:26.593180 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86c3742-cc15-45dc-80ff-a5c8744cd432 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:00:26.593861 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86c3742-cc15-45dc-80ff-a5c8744cd432 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:00:26.594141 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86c3742-cc15-45dc-80ff-a5c8744cd432 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:00:26.594407 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c86c3742-cc15-45dc-80ff-a5c8744cd432 None admin] PATH_INFO: `/v3/roles` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:00:26.595213 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c86c3742-cc15-45dc-80ff-a5c8744cd432 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 22:00:26.595513 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c86c3742-cc15-45dc-80ff-a5c8744cd432 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:00:26.596901 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c86c3742-cc15-45dc-80ff-a5c8744cd432 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:00:26.606737 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 538/1081] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Jul 13 22:00:26 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 22:00:26.612786 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fd5c3b-102f-4faf-a542-b3c5d2ea7ae3 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:00:26.718262 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91fd5c3b-102f-4faf-a542-b3c5d2ea7ae3 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:00:26.752646 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810b1843-e2cf-431f-9420-ab34b3d78594 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:00:26.753524 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810b1843-e2cf-431f-9420-ab34b3d78594 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:00:26.753800 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810b1843-e2cf-431f-9420-ab34b3d78594 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:00:26.754069 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-810b1843-e2cf-431f-9420-ab34b3d78594 None admin] PATH_INFO: `/v3/users/93d1c557bab54a69beb7295127463636` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:00:26.755308 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810b1843-e2cf-431f-9420-ab34b3d78594 None admin] RBAC: Authorizing `identity:get_user(user_id=93d1c557bab54a69beb7295127463636)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:00:26.756740 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-810b1843-e2cf-431f-9420-ab34b3d78594 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:00:26.758405 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 544/1082] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Jul 13 22:00:26 2022] GET /identity/v3/users/93d1c557bab54a69beb7295127463636 => generated 285 bytes in 146 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 22:00:26.764304 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86c3742-cc15-45dc-80ff-a5c8744cd432 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:00:26.860662 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:00:26.868003 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c86c3742-cc15-45dc-80ff-a5c8744cd432 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:00:26.900650 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199f161e-f6e2-4582-a8db-67b05d3799dd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:00:26.901507 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199f161e-f6e2-4582-a8db-67b05d3799dd None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:00:26.901783 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199f161e-f6e2-4582-a8db-67b05d3799dd None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:00:26.902050 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199f161e-f6e2-4582-a8db-67b05d3799dd None admin] PATH_INFO: `/v3/projects/3d8921c7315943f4a11901f37140ba1e` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:00:26.903195 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199f161e-f6e2-4582-a8db-67b05d3799dd None admin] RBAC: Authorizing `identity:get_project(project_id=3d8921c7315943f4a11901f37140ba1e)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:00:26.904631 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-199f161e-f6e2-4582-a8db-67b05d3799dd None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:00:26.906343 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 539/1083] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Jul 13 22:00:26 2022] GET /identity/v3/projects/3d8921c7315943f4a11901f37140ba1e => generated 311 bytes in 143 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 22:00:26.912337 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810b1843-e2cf-431f-9420-ab34b3d78594 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:00:26.971082 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:00:27.016505 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-810b1843-e2cf-431f-9420-ab34b3d78594 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:00:27.050974 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8283c5-33f8-4a91-9d90-5f8ca52ecf63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:00:27.052155 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8283c5-33f8-4a91-9d90-5f8ca52ecf63 None admin] REQUEST_METHOD: `PUT` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:00:27.052429 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8283c5-33f8-4a91-9d90-5f8ca52ecf63 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:00:27.052702 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8283c5-33f8-4a91-9d90-5f8ca52ecf63 None admin] PATH_INFO: `/v3/projects/3d8921c7315943f4a11901f37140ba1e/users/93d1c557bab54a69beb7295127463636/roles/44efba2579f342df8a0789c59d163ec3` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:00:27.054789 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8283c5-33f8-4a91-9d90-5f8ca52ecf63 None admin] RBAC: Authorizing `identity:create_grant(project_id=3d8921c7315943f4a11901f37140ba1e, user_id=93d1c557bab54a69beb7295127463636, role_id=44efba2579f342df8a0789c59d163ec3)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:00:27.056332 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae8283c5-33f8-4a91-9d90-5f8ca52ecf63 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:00:27.071147 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 545/1084] 10.222.0.52 () {58 vars in 1439 bytes} [Wed Jul 13 22:00:26 2022] PUT /identity/v3/projects/3d8921c7315943f4a11901f37140ba1e/users/93d1c557bab54a69beb7295127463636/roles/44efba2579f342df8a0789c59d163ec3 => generated 0 bytes in 159 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 22:00:28.497891 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fe5f11-aee8-4ae2-9586-4d65c7bb0313 None None] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:00:28.498526 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fe5f11-aee8-4ae2-9586-4d65c7bb0313 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:00:28.498526 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14fe5f11-aee8-4ae2-9586-4d65c7bb0313 None None] PATH_INFO: `/` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:00:28.499844 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 540/1085] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jul 13 22:00:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 22:00:28.505041 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7d8420-31d0-46a4-aa10-746c57419068 None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:00:28.505469 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7d8420-31d0-46a4-aa10-746c57419068 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:00:28.505576 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e7d8420-31d0-46a4-aa10-746c57419068 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:00:28.523055 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-5e7d8420-31d0-46a4-aa10-746c57419068 None None] MFA Rules not processed for user `cc99c93da5074cf5a48bc2aa9a92c5e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 22:00:28.524494 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-5e7d8420-31d0-46a4-aa10-746c57419068 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:00:28.598908 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 546/1086] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jul 13 22:00:28 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 13 22:00:28.622617 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14fe5f11-aee8-4ae2-9586-4d65c7bb0313 None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:00:28.732875 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14fe5f11-aee8-4ae2-9586-4d65c7bb0313 None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:00:28.768206 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b73f6aa-835a-41b1-993f-5ded26942c62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:00:28.768949 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b73f6aa-835a-41b1-993f-5ded26942c62 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:00:28.769229 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b73f6aa-835a-41b1-993f-5ded26942c62 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:00:28.769494 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b73f6aa-835a-41b1-993f-5ded26942c62 None admin] PATH_INFO: `/v3/roles/ResellerAdmin` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:00:28.780226 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b73f6aa-835a-41b1-993f-5ded26942c62 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 22:00:28.788627 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application [None req-0b73f6aa-835a-41b1-993f-5ded26942c62 None admin] Could not find role: ResellerAdmin.: keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 13 22:00:28.788627 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 22:00:28.788627 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 22:00:28.788627 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 22:00:28.788627 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 22:00:28.788627 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 22:00:28.788627 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 22:00:28.788627 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 22:00:28.788627 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 22:00:28.788627 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 22:00:28.788627 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 22:00:28.788627 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 22:00:28.788627 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 22:00:28.788627 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 22:00:28.788627 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 22:00:28.788627 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 22:00:28.789783 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 22:00:28.789783 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 22:00:28.789783 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 22:00:28.789783 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 22:00:28.789783 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 22:00:28.789783 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 22:00:28.789783 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 22:00:28.789783 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 22:00:28.789783 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 22:00:28.789783 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 22:00:28.789783 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 22:00:28.789783 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application with Lock( Jul 13 22:00:28.789783 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 22:00:28.789783 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._enter() Jul 13 22:00:28.789783 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 22:00:28.789783 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 22:00:28.790925 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 22:00:28.790925 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.creator() Jul 13 22:00:28.790925 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 22:00:28.790925 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application created_value = creator( Jul 13 22:00:28.790925 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 22:00:28.790925 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 22:00:28.790925 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 22:00:28.790925 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 22:00:28.790925 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 22:00:28.790925 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 22:00:28.790925 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: ResellerAdmin. Jul 13 22:00:28.790925 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Jul 13 22:00:28.790925 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 541/1087] 10.222.0.52 () {56 vars in 1120 bytes} [Wed Jul 13 22:00:28 2022] GET /identity/v3/roles/ResellerAdmin => generated 91 bytes in 168 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 22:00:28.796691 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7d8420-31d0-46a4-aa10-746c57419068 None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:00:28.845395 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:00:28.901786 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e7d8420-31d0-46a4-aa10-746c57419068 None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:00:28.939605 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19d6b83-de19-4105-8ef6-a5634188e604 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:00:28.940275 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19d6b83-de19-4105-8ef6-a5634188e604 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:00:28.940556 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19d6b83-de19-4105-8ef6-a5634188e604 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:00:28.940830 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19d6b83-de19-4105-8ef6-a5634188e604 None admin] PATH_INFO: `/v3/roles` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:00:28.941604 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19d6b83-de19-4105-8ef6-a5634188e604 None admin] RBAC: Adding query filter params (name=ResellerAdmin) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 22:00:28.941909 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19d6b83-de19-4105-8ef6-a5634188e604 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:00:28.943330 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19d6b83-de19-4105-8ef6-a5634188e604 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:00:28.952685 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 547/1088] 10.222.0.52 () {56 vars in 1115 bytes} [Wed Jul 13 22:00:28 2022] GET /identity/v3/roles?name=ResellerAdmin => generated 338 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 22:00:28.958479 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b73f6aa-835a-41b1-993f-5ded26942c62 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:00:28.985679 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:00:29.033735 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4780ef3-ee8e-4cbb-91be-1109a4466811 None None] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:00:29.033926 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4780ef3-ee8e-4cbb-91be-1109a4466811 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:00:29.034194 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4780ef3-ee8e-4cbb-91be-1109a4466811 None None] PATH_INFO: `/` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:00:29.035578 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 548/1089] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jul 13 22:00:29 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 22:00:29.040504 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611527db-7826-4db0-aaf5-469cbd9020db None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:00:29.040790 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611527db-7826-4db0-aaf5-469cbd9020db None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:00:29.041063 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-611527db-7826-4db0-aaf5-469cbd9020db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:00:29.058316 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-611527db-7826-4db0-aaf5-469cbd9020db None None] MFA Rules not processed for user `cc99c93da5074cf5a48bc2aa9a92c5e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 22:00:29.059534 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-611527db-7826-4db0-aaf5-469cbd9020db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:00:29.062514 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b73f6aa-835a-41b1-993f-5ded26942c62 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:00:29.098858 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13bfb70e-813a-4949-b312-3b46e47b95d7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:00:29.099658 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bfb70e-813a-4949-b312-3b46e47b95d7 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:00:29.099944 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bfb70e-813a-4949-b312-3b46e47b95d7 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:00:29.100219 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13bfb70e-813a-4949-b312-3b46e47b95d7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:00:29.111402 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13bfb70e-813a-4949-b312-3b46e47b95d7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:00:29.112857 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13bfb70e-813a-4949-b312-3b46e47b95d7 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:00:29.129052 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 549/1090] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jul 13 22:00:29 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 13 22:00:29.130143 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application [None req-13bfb70e-813a-4949-b312-3b46e47b95d7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 22:00:29.130143 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 22:00:29.130143 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 22:00:29.130143 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 22:00:29.130143 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 22:00:29.130143 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 22:00:29.130143 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 22:00:29.130143 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 22:00:29.130143 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 22:00:29.130143 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 22:00:29.130143 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 22:00:29.130143 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 22:00:29.130143 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 22:00:29.130143 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 22:00:29.130143 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 22:00:29.131199 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 22:00:29.131199 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 22:00:29.131199 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 22:00:29.131199 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 22:00:29.131199 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 22:00:29.131199 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 22:00:29.131199 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 22:00:29.131199 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 22:00:29.131199 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application with Lock( Jul 13 22:00:29.131199 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 22:00:29.131199 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._enter() Jul 13 22:00:29.131199 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 22:00:29.131199 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 22:00:29.131199 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 22:00:29.131199 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.creator() Jul 13 22:00:29.131199 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 22:00:29.131199 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application created_value = creator( Jul 13 22:00:29.132106 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 22:00:29.132106 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 22:00:29.132106 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 22:00:29.132106 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Jul 13 22:00:29.132106 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 542/1091] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Jul 13 22:00:28 2022] GET /identity/v3/domains/Default => generated 87 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 22:00:29.137567 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611527db-7826-4db0-aaf5-469cbd9020db None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:00:29.151575 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13bfb70e-813a-4949-b312-3b46e47b95d7 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:00:29.176970 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:00:29.233264 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:00:29.236790 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-611527db-7826-4db0-aaf5-469cbd9020db None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:00:29.256829 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13bfb70e-813a-4949-b312-3b46e47b95d7 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:00:29.270928 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3bbee0-2330-4192-8543-84b23ad919ea None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:00:29.271605 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3bbee0-2330-4192-8543-84b23ad919ea None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:00:29.271886 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3bbee0-2330-4192-8543-84b23ad919ea None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:00:29.272162 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa3bbee0-2330-4192-8543-84b23ad919ea None admin] PATH_INFO: `/v3/domains` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:00:29.272894 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3bbee0-2330-4192-8543-84b23ad919ea None admin] RBAC: Adding query filter params (name=Default) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 22:00:29.273241 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3bbee0-2330-4192-8543-84b23ad919ea None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:00:29.274728 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fa3bbee0-2330-4192-8543-84b23ad919ea None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:00:29.286806 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 550/1092] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Jul 13 22:00:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 22:00:29.290893 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de117c9f-4b10-425a-96a2-8e6db14e666b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:00:29.291778 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de117c9f-4b10-425a-96a2-8e6db14e666b None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:00:29.292081 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de117c9f-4b10-425a-96a2-8e6db14e666b None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:00:29.292369 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de117c9f-4b10-425a-96a2-8e6db14e666b None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:00:29.292731 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3bbee0-2330-4192-8543-84b23ad919ea None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:00:29.303193 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de117c9f-4b10-425a-96a2-8e6db14e666b None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 22:00:29.310791 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application [None req-de117c9f-4b10-425a-96a2-8e6db14e666b None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 13 22:00:29.310791 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 22:00:29.310791 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 22:00:29.310791 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 22:00:29.310791 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 22:00:29.310791 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 22:00:29.310791 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 22:00:29.310791 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 22:00:29.310791 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 22:00:29.310791 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 22:00:29.310791 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 22:00:29.310791 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 22:00:29.310791 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 22:00:29.310791 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 22:00:29.310791 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 22:00:29.310791 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 22:00:29.311946 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 22:00:29.311946 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 22:00:29.311946 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 22:00:29.311946 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 22:00:29.311946 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 22:00:29.311946 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 22:00:29.311946 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 22:00:29.311946 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 22:00:29.311946 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 22:00:29.311946 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 22:00:29.311946 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 22:00:29.311946 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application with Lock( Jul 13 22:00:29.311946 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 22:00:29.311946 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._enter() Jul 13 22:00:29.311946 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 22:00:29.311946 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 22:00:29.312815 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 22:00:29.312815 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.creator() Jul 13 22:00:29.312815 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 22:00:29.312815 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application created_value = creator( Jul 13 22:00:29.312815 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 22:00:29.312815 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 22:00:29.312815 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 22:00:29.312815 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 22:00:29.312815 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 22:00:29.312815 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 22:00:29.312815 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 13 22:00:29.312815 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Jul 13 22:00:29.312815 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 543/1093] 10.222.0.52 () {56 vars in 1096 bytes} [Wed Jul 13 22:00:29 2022] GET /identity/v3/roles/admin => generated 83 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 22:00:29.318040 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de117c9f-4b10-425a-96a2-8e6db14e666b None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:00:29.387960 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:00:29.391783 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa3bbee0-2330-4192-8543-84b23ad919ea None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:00:29.418114 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de117c9f-4b10-425a-96a2-8e6db14e666b None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:00:29.425297 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33402020-db29-4092-9623-405d499f4ee4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:00:29.426098 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33402020-db29-4092-9623-405d499f4ee4 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:00:29.426371 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33402020-db29-4092-9623-405d499f4ee4 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:00:29.426674 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33402020-db29-4092-9623-405d499f4ee4 None admin] PATH_INFO: `/v3/users/nova` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:00:29.434202 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33402020-db29-4092-9623-405d499f4ee4 None admin] RBAC: Authorizing `identity:get_user(user_id=nova)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:00:29.435675 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33402020-db29-4092-9623-405d499f4ee4 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:00:29.450350 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application [None req-33402020-db29-4092-9623-405d499f4ee4 None admin] Could not find user: nova.: keystone.exception.UserNotFound: Could not find user: nova. Jul 13 22:00:29.450350 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 22:00:29.450350 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 22:00:29.450350 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 22:00:29.450350 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 22:00:29.450350 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 22:00:29.450350 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 22:00:29.450350 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 22:00:29.450350 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 22:00:29.450350 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 22:00:29.450350 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 22:00:29.450350 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 22:00:29.450350 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 13 22:00:29.450350 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 13 22:00:29.450350 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 13 22:00:29.450350 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 13 22:00:29.451343 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 22:00:29.451343 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 22:00:29.451343 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 13 22:00:29.451343 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 22:00:29.451343 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 13 22:00:29.451343 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 22:00:29.451343 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 22:00:29.451343 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 22:00:29.451343 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 22:00:29.451343 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 22:00:29.451343 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 22:00:29.451343 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application with Lock( Jul 13 22:00:29.451343 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 22:00:29.451343 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._enter() Jul 13 22:00:29.451343 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 22:00:29.451343 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 22:00:29.451343 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 22:00:29.451343 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.creator() Jul 13 22:00:29.452351 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 22:00:29.452351 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application created_value = creator( Jul 13 22:00:29.452351 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 13 22:00:29.452351 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 13 22:00:29.452351 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 13 22:00:29.452351 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 13 22:00:29.452351 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 13 22:00:29.452351 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 13 22:00:29.452351 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: nova. Jul 13 22:00:29.452351 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Jul 13 22:00:29.452900 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88495832-b901-44a9-97ba-76a4c9c56977 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:00:29.452992 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 551/1094] 10.222.0.52 () {56 vars in 1093 bytes} [Wed Jul 13 22:00:29 2022] GET /identity/v3/users/nova => generated 82 bytes in 159 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 13 22:00:29.453063 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88495832-b901-44a9-97ba-76a4c9c56977 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:00:29.453063 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88495832-b901-44a9-97ba-76a4c9c56977 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:00:29.453063 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88495832-b901-44a9-97ba-76a4c9c56977 None admin] PATH_INFO: `/v3/roles` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:00:29.453559 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88495832-b901-44a9-97ba-76a4c9c56977 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 22:00:29.453763 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88495832-b901-44a9-97ba-76a4c9c56977 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:00:29.455144 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-88495832-b901-44a9-97ba-76a4c9c56977 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:00:29.457466 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33402020-db29-4092-9623-405d499f4ee4 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:00:29.457603 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:00:29.469334 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 544/1095] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Jul 13 22:00:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 22:00:29.470318 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88495832-b901-44a9-97ba-76a4c9c56977 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:00:29.560302 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33402020-db29-4092-9623-405d499f4ee4 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:00:29.570880 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88495832-b901-44a9-97ba-76a4c9c56977 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:00:29.595663 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739347af-9c01-4d33-acbe-fd92412b6a66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:00:29.596351 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739347af-9c01-4d33-acbe-fd92412b6a66 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:00:29.596636 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739347af-9c01-4d33-acbe-fd92412b6a66 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:00:29.596912 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-739347af-9c01-4d33-acbe-fd92412b6a66 None admin] PATH_INFO: `/v3/users` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:00:29.597788 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739347af-9c01-4d33-acbe-fd92412b6a66 None admin] RBAC: Adding query filter params (domain_id=default, name=nova) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 22:00:29.598094 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739347af-9c01-4d33-acbe-fd92412b6a66 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:00:29.599596 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-739347af-9c01-4d33-acbe-fd92412b6a66 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:00:29.605295 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276fcdd7-9ebc-491c-9f24-cd3ce79b5355 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:00:29.606174 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276fcdd7-9ebc-491c-9f24-cd3ce79b5355 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:00:29.606458 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276fcdd7-9ebc-491c-9f24-cd3ce79b5355 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:00:29.606754 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-276fcdd7-9ebc-491c-9f24-cd3ce79b5355 None admin] PATH_INFO: `/v3/users/93d1c557bab54a69beb7295127463636` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:00:29.607941 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276fcdd7-9ebc-491c-9f24-cd3ce79b5355 None admin] RBAC: Authorizing `identity:get_user(user_id=93d1c557bab54a69beb7295127463636)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:00:29.609382 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-276fcdd7-9ebc-491c-9f24-cd3ce79b5355 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:00:29.611037 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 545/1096] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Jul 13 22:00:29 2022] GET /identity/v3/users/93d1c557bab54a69beb7295127463636 => generated 285 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 22:00:29.616720 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276fcdd7-9ebc-491c-9f24-cd3ce79b5355 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:00:29.621650 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 552/1097] 10.222.0.52 () {56 vars in 1133 bytes} [Wed Jul 13 22:00:29 2022] GET /identity/v3/users?domain_id=default&name=nova => generated 368 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 22:00:29.622524 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739347af-9c01-4d33-acbe-fd92412b6a66 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:00:29.639796 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:00:29.681521 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:00:29.714592 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-276fcdd7-9ebc-491c-9f24-cd3ce79b5355 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:00:29.719981 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-739347af-9c01-4d33-acbe-fd92412b6a66 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:00:29.746712 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e47645-f0d0-48ec-87ef-e701b0e2c137 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:00:29.747359 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e47645-f0d0-48ec-87ef-e701b0e2c137 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:00:29.747632 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e47645-f0d0-48ec-87ef-e701b0e2c137 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:00:29.747902 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-68e47645-f0d0-48ec-87ef-e701b0e2c137 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:00:29.753516 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856e1e4c-3d90-4da1-aa7c-4dc2d36a9ea9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:00:29.754175 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856e1e4c-3d90-4da1-aa7c-4dc2d36a9ea9 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:00:29.754436 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856e1e4c-3d90-4da1-aa7c-4dc2d36a9ea9 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:00:29.754814 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-856e1e4c-3d90-4da1-aa7c-4dc2d36a9ea9 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:00:29.816901 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e47645-f0d0-48ec-87ef-e701b0e2c137 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:00:29.818261 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-68e47645-f0d0-48ec-87ef-e701b0e2c137 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:00:29.820436 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856e1e4c-3d90-4da1-aa7c-4dc2d36a9ea9 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:00:29.821885 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-856e1e4c-3d90-4da1-aa7c-4dc2d36a9ea9 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:00:29.840343 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:00:29.884698 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:00:29.919110 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 546/1098] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Jul 13 22:00:29 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 303 msecs (HTTP/1.1 200) 6 headers in 359 bytes (2 switches on core 0) Jul 13 22:00:29.927175 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 553/1099] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Jul 13 22:00:29 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 300 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 13 22:00:29.927320 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e47645-f0d0-48ec-87ef-e701b0e2c137 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:00:29.928165 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856e1e4c-3d90-4da1-aa7c-4dc2d36a9ea9 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:00:30.019982 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-68e47645-f0d0-48ec-87ef-e701b0e2c137 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:00:30.023444 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-856e1e4c-3d90-4da1-aa7c-4dc2d36a9ea9 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:00:30.039990 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:00:30.054930 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc12236-be79-4897-8413-e8d0ee46920b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:00:30.055795 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc12236-be79-4897-8413-e8d0ee46920b None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:00:30.056071 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc12236-be79-4897-8413-e8d0ee46920b None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:00:30.056341 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcc12236-be79-4897-8413-e8d0ee46920b None admin] PATH_INFO: `/v3/projects/3d8921c7315943f4a11901f37140ba1e` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:00:30.057498 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc12236-be79-4897-8413-e8d0ee46920b None admin] RBAC: Authorizing `identity:get_project(project_id=3d8921c7315943f4a11901f37140ba1e)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:00:30.058508 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e532cb5-24d8-49ff-b93c-6de3a4df4992 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:00:30.058961 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dcc12236-be79-4897-8413-e8d0ee46920b None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:00:30.059317 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e532cb5-24d8-49ff-b93c-6de3a4df4992 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:00:30.059596 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e532cb5-24d8-49ff-b93c-6de3a4df4992 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:00:30.059874 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e532cb5-24d8-49ff-b93c-6de3a4df4992 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:00:30.060857 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 547/1100] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Jul 13 22:00:29 2022] GET /identity/v3/projects/3d8921c7315943f4a11901f37140ba1e => generated 311 bytes in 136 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 22:00:30.066707 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc12236-be79-4897-8413-e8d0ee46920b None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:00:30.070860 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e532cb5-24d8-49ff-b93c-6de3a4df4992 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:00:30.072263 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e532cb5-24d8-49ff-b93c-6de3a4df4992 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:00:30.091097 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application [None req-2e532cb5-24d8-49ff-b93c-6de3a4df4992 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 22:00:30.091097 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 22:00:30.091097 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 22:00:30.091097 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 22:00:30.091097 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 22:00:30.091097 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 22:00:30.091097 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 22:00:30.091097 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 22:00:30.091097 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 22:00:30.091097 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 22:00:30.091097 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 22:00:30.091097 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 22:00:30.091097 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 22:00:30.091097 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 22:00:30.091097 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 22:00:30.092023 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 22:00:30.092023 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 22:00:30.092023 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 22:00:30.092023 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 22:00:30.092023 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 22:00:30.092023 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 22:00:30.092023 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 22:00:30.092023 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 22:00:30.092023 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application with Lock( Jul 13 22:00:30.092023 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 22:00:30.092023 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._enter() Jul 13 22:00:30.092023 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 22:00:30.092023 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 22:00:30.092023 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 22:00:30.092023 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.creator() Jul 13 22:00:30.092023 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 22:00:30.092023 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application created_value = creator( Jul 13 22:00:30.092939 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 22:00:30.092939 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 22:00:30.092939 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 22:00:30.092939 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Jul 13 22:00:30.092939 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 554/1101] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Jul 13 22:00:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 22:00:30.098090 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e532cb5-24d8-49ff-b93c-6de3a4df4992 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:00:30.100010 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:00:30.165184 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcc12236-be79-4897-8413-e8d0ee46920b None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:00:30.195713 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e532cb5-24d8-49ff-b93c-6de3a4df4992 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:00:30.197563 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3bc803-f104-4cea-8ffa-20f999f2a37c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:00:30.198221 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3bc803-f104-4cea-8ffa-20f999f2a37c None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:00:30.198518 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3bc803-f104-4cea-8ffa-20f999f2a37c None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:00:30.198784 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a3bc803-f104-4cea-8ffa-20f999f2a37c None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:00:30.199961 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3bc803-f104-4cea-8ffa-20f999f2a37c None admin] RBAC: Adding query filter params (role.id=44efba2579f342df8a0789c59d163ec3, scope.project.id=3d8921c7315943f4a11901f37140ba1e, user.id=93d1c557bab54a69beb7295127463636) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 22:00:30.200262 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3bc803-f104-4cea-8ffa-20f999f2a37c None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:00:30.201682 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a3bc803-f104-4cea-8ffa-20f999f2a37c None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:00:30.207657 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 548/1102] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Jul 13 22:00:30 2022] GET /identity/v3/role_assignments?user.id=93d1c557bab54a69beb7295127463636&scope.project.id=3d8921c7315943f4a11901f37140ba1e&role.id=44efba2579f342df8a0789c59d163ec3 => generated 608 bytes in 141 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 22:00:30.231073 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067b71cc-f32d-4c90-80ac-44ca58546f63 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:00:30.231770 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067b71cc-f32d-4c90-80ac-44ca58546f63 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:00:30.232052 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067b71cc-f32d-4c90-80ac-44ca58546f63 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:00:30.232326 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-067b71cc-f32d-4c90-80ac-44ca58546f63 None admin] PATH_INFO: `/v3/domains` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:00:30.233080 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067b71cc-f32d-4c90-80ac-44ca58546f63 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 22:00:30.233385 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067b71cc-f32d-4c90-80ac-44ca58546f63 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:00:30.234866 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-067b71cc-f32d-4c90-80ac-44ca58546f63 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:00:30.248109 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 555/1103] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Jul 13 22:00:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 22:00:30.254157 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3bc803-f104-4cea-8ffa-20f999f2a37c None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:00:30.360521 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:00:30.365733 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a3bc803-f104-4cea-8ffa-20f999f2a37c None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:00:30.402109 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999e3150-03d9-4baf-bb09-d5f55aeaa3cc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:00:30.402946 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999e3150-03d9-4baf-bb09-d5f55aeaa3cc None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:00:30.403228 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999e3150-03d9-4baf-bb09-d5f55aeaa3cc None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:00:30.403501 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999e3150-03d9-4baf-bb09-d5f55aeaa3cc None admin] PATH_INFO: `/v3/projects/service` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:00:30.410335 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999e3150-03d9-4baf-bb09-d5f55aeaa3cc None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:00:30.411820 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-999e3150-03d9-4baf-bb09-d5f55aeaa3cc None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:00:30.426286 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application [None req-999e3150-03d9-4baf-bb09-d5f55aeaa3cc None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 13 22:00:30.426286 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 22:00:30.426286 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 22:00:30.426286 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 22:00:30.426286 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 22:00:30.426286 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 22:00:30.426286 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 22:00:30.426286 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 22:00:30.426286 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 22:00:30.426286 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 22:00:30.426286 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 22:00:30.426286 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 22:00:30.426286 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 13 22:00:30.426286 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 13 22:00:30.426286 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 13 22:00:30.427318 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 13 22:00:30.427318 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 22:00:30.427318 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 22:00:30.427318 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 22:00:30.427318 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 22:00:30.427318 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 22:00:30.427318 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 22:00:30.427318 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 22:00:30.427318 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application with Lock( Jul 13 22:00:30.427318 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 22:00:30.427318 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._enter() Jul 13 22:00:30.427318 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 22:00:30.427318 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 22:00:30.427318 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 22:00:30.427318 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.creator() Jul 13 22:00:30.427318 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 22:00:30.427318 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application created_value = creator( Jul 13 22:00:30.428214 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 13 22:00:30.428214 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 13 22:00:30.428214 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 13 22:00:30.428214 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 13 22:00:30.428214 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 13 22:00:30.428214 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 13 22:00:30.428214 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 13 22:00:30.428214 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Jul 13 22:00:30.428214 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 549/1104] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Jul 13 22:00:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 22:00:30.433569 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067b71cc-f32d-4c90-80ac-44ca58546f63 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:00:30.463790 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:00:30.537763 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-067b71cc-f32d-4c90-80ac-44ca58546f63 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:00:30.574843 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d618d7ed-0e9d-482e-8540-abc2a5bd0bb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:00:30.575479 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d618d7ed-0e9d-482e-8540-abc2a5bd0bb2 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:00:30.575774 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d618d7ed-0e9d-482e-8540-abc2a5bd0bb2 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:00:30.576036 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d618d7ed-0e9d-482e-8540-abc2a5bd0bb2 None admin] PATH_INFO: `/v3/projects` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:00:30.576816 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d618d7ed-0e9d-482e-8540-abc2a5bd0bb2 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 22:00:30.577150 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d618d7ed-0e9d-482e-8540-abc2a5bd0bb2 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:00:30.578610 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d618d7ed-0e9d-482e-8540-abc2a5bd0bb2 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:00:30.591134 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 556/1105] 10.222.0.52 () {56 vars in 1148 bytes} [Wed Jul 13 22:00:30 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 22:00:30.597274 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999e3150-03d9-4baf-bb09-d5f55aeaa3cc None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:00:30.703694 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-999e3150-03d9-4baf-bb09-d5f55aeaa3cc None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:00:30.738749 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3269f056-11c3-4237-9c21-5f0b3954ff17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:00:30.739400 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3269f056-11c3-4237-9c21-5f0b3954ff17 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:00:30.739675 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3269f056-11c3-4237-9c21-5f0b3954ff17 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:00:30.739945 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3269f056-11c3-4237-9c21-5f0b3954ff17 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:00:30.740778 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3269f056-11c3-4237-9c21-5f0b3954ff17 None admin] RBAC: Adding query filter params (role.id=3987457f5fe64b568601d117c2c0ba9d, scope.project.id=417e3e2f1333481795a089bef2e24bea, user.id=f84281091687464990ffbca590b9343b) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 22:00:30.741087 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3269f056-11c3-4237-9c21-5f0b3954ff17 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:00:30.742522 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3269f056-11c3-4237-9c21-5f0b3954ff17 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:00:30.748498 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 550/1106] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Jul 13 22:00:30 2022] GET /identity/v3/role_assignments?user.id=f84281091687464990ffbca590b9343b&scope.project.id=417e3e2f1333481795a089bef2e24bea&role.id=3987457f5fe64b568601d117c2c0ba9d => generated 608 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 22:00:32.306740 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3fc341-874c-4dee-88bd-97c4d9ac3e12 None None] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:00:32.307632 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3fc341-874c-4dee-88bd-97c4d9ac3e12 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:00:32.307632 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ee3fc341-874c-4dee-88bd-97c4d9ac3e12 None None] PATH_INFO: `/` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:00:32.309789 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 557/1107] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jul 13 22:00:32 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 22:00:32.316272 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8921fa55-bdea-47a7-b451-afd3c2f9ace5 None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:00:32.316732 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8921fa55-bdea-47a7-b451-afd3c2f9ace5 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:00:32.316837 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8921fa55-bdea-47a7-b451-afd3c2f9ace5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:00:32.336499 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-8921fa55-bdea-47a7-b451-afd3c2f9ace5 None None] MFA Rules not processed for user `cc99c93da5074cf5a48bc2aa9a92c5e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 22:00:32.337704 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-8921fa55-bdea-47a7-b451-afd3c2f9ace5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:00:32.355150 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:00:32.411647 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 551/1108] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jul 13 22:00:32 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 13 22:00:32.450555 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3fc341-874c-4dee-88bd-97c4d9ac3e12 None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:00:32.563775 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:00:32.578753 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ee3fc341-874c-4dee-88bd-97c4d9ac3e12 None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:00:32.613320 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e20aeebd-bb65-41da-807e-5a872c316c3b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:00:32.614119 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20aeebd-bb65-41da-807e-5a872c316c3b None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:00:32.614401 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20aeebd-bb65-41da-807e-5a872c316c3b None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:00:32.614709 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e20aeebd-bb65-41da-807e-5a872c316c3b None admin] PATH_INFO: `/v3/domains/default` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:00:32.615931 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20aeebd-bb65-41da-807e-5a872c316c3b None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:00:32.617426 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e20aeebd-bb65-41da-807e-5a872c316c3b None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:00:32.619085 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 558/1109] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Jul 13 22:00:32 2022] GET /identity/v3/domains/default => generated 199 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 22:00:32.625020 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8921fa55-bdea-47a7-b451-afd3c2f9ace5 None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:00:32.732764 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8921fa55-bdea-47a7-b451-afd3c2f9ace5 None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:00:32.776693 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7043c59-5e27-4819-a8ee-850d5e0a5ea9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:00:32.777361 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7043c59-5e27-4819-a8ee-850d5e0a5ea9 None admin] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:00:32.777635 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7043c59-5e27-4819-a8ee-850d5e0a5ea9 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:00:32.777903 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7043c59-5e27-4819-a8ee-850d5e0a5ea9 None admin] PATH_INFO: `/v3/users` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:00:32.778640 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7043c59-5e27-4819-a8ee-850d5e0a5ea9 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:00:32.780076 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7043c59-5e27-4819-a8ee-850d5e0a5ea9 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:00:32.805656 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 552/1110] 10.222.0.52 () {60 vars in 1132 bytes} [Wed Jul 13 22:00:32 2022] POST /identity/v3/users => generated 286 bytes in 181 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 22:00:34.885313 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8956cc87-0761-43c5-914a-829af1badcd4 None None] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:00:34.885844 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8956cc87-0761-43c5-914a-829af1badcd4 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:00:34.885844 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8956cc87-0761-43c5-914a-829af1badcd4 None None] PATH_INFO: `/` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:00:34.887151 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 559/1111] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jul 13 22:00:34 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 22:00:34.893557 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1e5879-19d4-4e50-bb88-656d6f947302 None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:00:34.894286 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1e5879-19d4-4e50-bb88-656d6f947302 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:00:34.894434 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d1e5879-19d4-4e50-bb88-656d6f947302 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:00:34.915974 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-0d1e5879-19d4-4e50-bb88-656d6f947302 None None] MFA Rules not processed for user `cc99c93da5074cf5a48bc2aa9a92c5e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 22:00:34.917157 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-0d1e5879-19d4-4e50-bb88-656d6f947302 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:00:34.919290 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:00:34.991629 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 553/1112] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jul 13 22:00:34 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 13 22:00:35.027839 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8956cc87-0761-43c5-914a-829af1badcd4 None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:00:35.143933 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8956cc87-0761-43c5-914a-829af1badcd4 None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:00:35.183449 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469f4030-698e-4754-8172-58088d0ceac1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:00:35.184697 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469f4030-698e-4754-8172-58088d0ceac1 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:00:35.185076 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469f4030-698e-4754-8172-58088d0ceac1 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:00:35.185433 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-469f4030-698e-4754-8172-58088d0ceac1 None admin] PATH_INFO: `/v3/roles/db84c6c897534af1a60b9ce62532f383` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:00:35.188048 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469f4030-698e-4754-8172-58088d0ceac1 None admin] RBAC: Authorizing `identity:get_role(role_id=db84c6c897534af1a60b9ce62532f383)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:00:35.190193 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-469f4030-698e-4754-8172-58088d0ceac1 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:00:35.192377 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 560/1113] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Jul 13 22:00:35 2022] GET /identity/v3/roles/db84c6c897534af1a60b9ce62532f383 => generated 223 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 22:00:35.198388 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1e5879-19d4-4e50-bb88-656d6f947302 None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:00:35.302379 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d1e5879-19d4-4e50-bb88-656d6f947302 None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:00:35.339953 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b214a6-a508-454f-a916-04569b4aa10c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:00:35.340813 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b214a6-a508-454f-a916-04569b4aa10c None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:00:35.341082 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b214a6-a508-454f-a916-04569b4aa10c None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:00:35.341347 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40b214a6-a508-454f-a916-04569b4aa10c None admin] PATH_INFO: `/v3/users/5d882bf5681b4b15ba780ba16b0e7106` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:00:35.349001 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:00:35.356177 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b214a6-a508-454f-a916-04569b4aa10c None admin] RBAC: Authorizing `identity:get_user(user_id=5d882bf5681b4b15ba780ba16b0e7106)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:00:35.357623 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40b214a6-a508-454f-a916-04569b4aa10c None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:00:35.359295 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 554/1114] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Jul 13 22:00:35 2022] GET /identity/v3/users/5d882bf5681b4b15ba780ba16b0e7106 => generated 286 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 22:00:35.366047 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469f4030-698e-4754-8172-58088d0ceac1 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:00:35.402003 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:00:35.484781 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-469f4030-698e-4754-8172-58088d0ceac1 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:00:35.522932 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43037c51-384b-4ce2-99cc-760338996949 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:00:35.523787 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43037c51-384b-4ce2-99cc-760338996949 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:00:35.524066 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43037c51-384b-4ce2-99cc-760338996949 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:00:35.524334 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43037c51-384b-4ce2-99cc-760338996949 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:00:35.600447 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43037c51-384b-4ce2-99cc-760338996949 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:00:35.601938 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43037c51-384b-4ce2-99cc-760338996949 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:00:35.627436 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:00:35.714817 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 561/1115] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Jul 13 22:00:35 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 350 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 13 22:00:35.721711 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b214a6-a508-454f-a916-04569b4aa10c None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:00:35.831119 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40b214a6-a508-454f-a916-04569b4aa10c None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:00:35.870672 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18e14cb8-6070-4762-b01b-2c53a1f4b18d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:00:35.871529 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e14cb8-6070-4762-b01b-2c53a1f4b18d None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:00:35.871810 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e14cb8-6070-4762-b01b-2c53a1f4b18d None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:00:35.872078 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18e14cb8-6070-4762-b01b-2c53a1f4b18d None admin] PATH_INFO: `/v3/projects/3d8921c7315943f4a11901f37140ba1e` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:00:35.873537 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e14cb8-6070-4762-b01b-2c53a1f4b18d None admin] RBAC: Authorizing `identity:get_project(project_id=3d8921c7315943f4a11901f37140ba1e)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:00:35.875020 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18e14cb8-6070-4762-b01b-2c53a1f4b18d None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:00:35.876738 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 555/1116] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Jul 13 22:00:35 2022] GET /identity/v3/projects/3d8921c7315943f4a11901f37140ba1e => generated 311 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 22:00:35.883399 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43037c51-384b-4ce2-99cc-760338996949 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:00:36.002904 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43037c51-384b-4ce2-99cc-760338996949 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:00:36.026588 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:00:36.038938 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-16c63675-3630-4e95-b5fe-78b8175c8858 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:00:36.039588 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c63675-3630-4e95-b5fe-78b8175c8858 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:00:36.039865 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c63675-3630-4e95-b5fe-78b8175c8858 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:00:36.040137 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-16c63675-3630-4e95-b5fe-78b8175c8858 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:00:36.040962 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c63675-3630-4e95-b5fe-78b8175c8858 None admin] RBAC: Adding query filter params (role.id=db84c6c897534af1a60b9ce62532f383, scope.project.id=3d8921c7315943f4a11901f37140ba1e, user.id=5d882bf5681b4b15ba780ba16b0e7106) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 22:00:36.041265 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c63675-3630-4e95-b5fe-78b8175c8858 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:00:36.042716 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-16c63675-3630-4e95-b5fe-78b8175c8858 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:00:36.048567 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 562/1117] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Jul 13 22:00:35 2022] GET /identity/v3/role_assignments?user.id=5d882bf5681b4b15ba780ba16b0e7106&scope.project.id=3d8921c7315943f4a11901f37140ba1e&role.id=db84c6c897534af1a60b9ce62532f383 => generated 259 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 22:00:38.028926 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417e93cc-9d95-4207-a45a-f9804066bae3 None None] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:00:38.029570 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417e93cc-9d95-4207-a45a-f9804066bae3 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:00:38.029570 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-417e93cc-9d95-4207-a45a-f9804066bae3 None None] PATH_INFO: `/` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:00:38.030907 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 556/1118] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jul 13 22:00:38 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 22:00:38.037241 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068017ff-916a-4236-aa9e-bc681407eacb None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:00:38.038002 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068017ff-916a-4236-aa9e-bc681407eacb None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:00:38.038145 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-068017ff-916a-4236-aa9e-bc681407eacb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:00:38.062517 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-068017ff-916a-4236-aa9e-bc681407eacb None None] MFA Rules not processed for user `cc99c93da5074cf5a48bc2aa9a92c5e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 22:00:38.063699 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-068017ff-916a-4236-aa9e-bc681407eacb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:00:38.138505 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 563/1119] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jul 13 22:00:38 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 13 22:00:38.162314 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417e93cc-9d95-4207-a45a-f9804066bae3 None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:00:38.235287 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:00:38.278787 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-417e93cc-9d95-4207-a45a-f9804066bae3 None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:00:38.314418 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062b2b43-7f90-4e32-b26d-478abf52d8e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:00:38.315238 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062b2b43-7f90-4e32-b26d-478abf52d8e7 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:00:38.315516 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062b2b43-7f90-4e32-b26d-478abf52d8e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:00:38.315786 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-062b2b43-7f90-4e32-b26d-478abf52d8e7 None admin] PATH_INFO: `/v3/roles/db84c6c897534af1a60b9ce62532f383` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:00:38.317313 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-062b2b43-7f90-4e32-b26d-478abf52d8e7 None admin] RBAC: Authorizing `identity:get_role(role_id=db84c6c897534af1a60b9ce62532f383)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:00:38.318713 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-062b2b43-7f90-4e32-b26d-478abf52d8e7 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:00:38.320595 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 557/1120] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Jul 13 22:00:38 2022] GET /identity/v3/roles/db84c6c897534af1a60b9ce62532f383 => generated 223 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 22:00:38.327628 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068017ff-916a-4236-aa9e-bc681407eacb None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:00:38.438039 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-068017ff-916a-4236-aa9e-bc681407eacb None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:00:38.456299 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:00:38.475097 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8e8c53-8d4c-41f4-9aa6-f87d59a143fa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:00:38.475971 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8e8c53-8d4c-41f4-9aa6-f87d59a143fa None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:00:38.476312 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8e8c53-8d4c-41f4-9aa6-f87d59a143fa None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:00:38.476579 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e8e8c53-8d4c-41f4-9aa6-f87d59a143fa None admin] PATH_INFO: `/v3/users/5d882bf5681b4b15ba780ba16b0e7106` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:00:38.477815 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8e8c53-8d4c-41f4-9aa6-f87d59a143fa None admin] RBAC: Authorizing `identity:get_user(user_id=5d882bf5681b4b15ba780ba16b0e7106)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:00:38.479290 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1e8e8c53-8d4c-41f4-9aa6-f87d59a143fa None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:00:38.481052 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 564/1121] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Jul 13 22:00:38 2022] GET /identity/v3/users/5d882bf5681b4b15ba780ba16b0e7106 => generated 286 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 22:00:38.487068 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062b2b43-7f90-4e32-b26d-478abf52d8e7 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:00:38.597297 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-062b2b43-7f90-4e32-b26d-478abf52d8e7 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:00:38.628062 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:00:38.634268 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65bc26a8-f863-43b3-b15d-ec446b4487b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:00:38.635253 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bc26a8-f863-43b3-b15d-ec446b4487b2 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:00:38.635596 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bc26a8-f863-43b3-b15d-ec446b4487b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:00:38.635952 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65bc26a8-f863-43b3-b15d-ec446b4487b2 None admin] PATH_INFO: `/v3/projects/3d8921c7315943f4a11901f37140ba1e` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:00:38.637203 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65bc26a8-f863-43b3-b15d-ec446b4487b2 None admin] RBAC: Authorizing `identity:get_project(project_id=3d8921c7315943f4a11901f37140ba1e)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:00:38.638747 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65bc26a8-f863-43b3-b15d-ec446b4487b2 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:00:38.640500 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 558/1122] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Jul 13 22:00:38 2022] GET /identity/v3/projects/3d8921c7315943f4a11901f37140ba1e => generated 311 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 22:00:38.646773 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8e8c53-8d4c-41f4-9aa6-f87d59a143fa None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:00:38.746104 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e8e8c53-8d4c-41f4-9aa6-f87d59a143fa None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:00:38.782173 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51984be0-e8f3-4f54-94dd-b9779edc1501 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:00:38.783130 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51984be0-e8f3-4f54-94dd-b9779edc1501 None admin] REQUEST_METHOD: `PUT` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:00:38.783408 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51984be0-e8f3-4f54-94dd-b9779edc1501 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:00:38.783688 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51984be0-e8f3-4f54-94dd-b9779edc1501 None admin] PATH_INFO: `/v3/projects/3d8921c7315943f4a11901f37140ba1e/users/5d882bf5681b4b15ba780ba16b0e7106/roles/db84c6c897534af1a60b9ce62532f383` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:00:38.786006 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51984be0-e8f3-4f54-94dd-b9779edc1501 None admin] RBAC: Authorizing `identity:create_grant(project_id=3d8921c7315943f4a11901f37140ba1e, user_id=5d882bf5681b4b15ba780ba16b0e7106, role_id=db84c6c897534af1a60b9ce62532f383)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:00:38.787492 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51984be0-e8f3-4f54-94dd-b9779edc1501 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:00:38.804429 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 565/1123] 10.222.0.52 () {58 vars in 1439 bytes} [Wed Jul 13 22:00:38 2022] PUT /identity/v3/projects/3d8921c7315943f4a11901f37140ba1e/users/5d882bf5681b4b15ba780ba16b0e7106/roles/db84c6c897534af1a60b9ce62532f383 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 22:00:40.769070 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f923b8a9-e1b3-4971-ba7a-1d7962895f32 None None] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:00:40.769646 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f923b8a9-e1b3-4971-ba7a-1d7962895f32 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:00:40.769646 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f923b8a9-e1b3-4971-ba7a-1d7962895f32 None None] PATH_INFO: `/` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:00:40.771039 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 559/1124] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jul 13 22:00:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 22:00:40.777359 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea7c140-2d62-46e0-a81a-5fe08d5e243c None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:00:40.777809 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea7c140-2d62-46e0-a81a-5fe08d5e243c None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:00:40.777916 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ea7c140-2d62-46e0-a81a-5fe08d5e243c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:00:40.799542 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-6ea7c140-2d62-46e0-a81a-5fe08d5e243c None None] MFA Rules not processed for user `cc99c93da5074cf5a48bc2aa9a92c5e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 22:00:40.800731 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-6ea7c140-2d62-46e0-a81a-5fe08d5e243c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:00:40.844673 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:00:40.875568 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 566/1125] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jul 13 22:00:40 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 13 22:00:40.909016 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f923b8a9-e1b3-4971-ba7a-1d7962895f32 None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:00:41.022783 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f923b8a9-e1b3-4971-ba7a-1d7962895f32 None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:00:41.058191 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a727052-a955-4d3f-89c5-51209eb0d676 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:00:41.059065 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a727052-a955-4d3f-89c5-51209eb0d676 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:00:41.059346 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a727052-a955-4d3f-89c5-51209eb0d676 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:00:41.059607 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a727052-a955-4d3f-89c5-51209eb0d676 None admin] PATH_INFO: `/v3/roles/db84c6c897534af1a60b9ce62532f383` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:00:41.061195 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a727052-a955-4d3f-89c5-51209eb0d676 None admin] RBAC: Authorizing `identity:get_role(role_id=db84c6c897534af1a60b9ce62532f383)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:00:41.062592 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a727052-a955-4d3f-89c5-51209eb0d676 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:00:41.063928 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 560/1126] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Jul 13 22:00:40 2022] GET /identity/v3/roles/db84c6c897534af1a60b9ce62532f383 => generated 223 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 22:00:41.073180 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea7c140-2d62-46e0-a81a-5fe08d5e243c None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:00:41.189777 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ea7c140-2d62-46e0-a81a-5fe08d5e243c None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:00:41.226105 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4941631d-e5ca-4263-9117-3113faf0f2ca None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:00:41.227077 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4941631d-e5ca-4263-9117-3113faf0f2ca None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:00:41.227357 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4941631d-e5ca-4263-9117-3113faf0f2ca None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:00:41.227625 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4941631d-e5ca-4263-9117-3113faf0f2ca None admin] PATH_INFO: `/v3/users/5d882bf5681b4b15ba780ba16b0e7106` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:00:41.228945 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4941631d-e5ca-4263-9117-3113faf0f2ca None admin] RBAC: Authorizing `identity:get_user(user_id=5d882bf5681b4b15ba780ba16b0e7106)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:00:41.230397 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4941631d-e5ca-4263-9117-3113faf0f2ca None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:00:41.232049 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 567/1127] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Jul 13 22:00:41 2022] GET /identity/v3/users/5d882bf5681b4b15ba780ba16b0e7106 => generated 286 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 22:00:41.238760 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a727052-a955-4d3f-89c5-51209eb0d676 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:00:41.319121 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:00:41.356837 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a727052-a955-4d3f-89c5-51209eb0d676 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:00:41.392465 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc005c66-452f-463c-90ab-a82f9f0ebc43 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:00:41.393109 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc005c66-452f-463c-90ab-a82f9f0ebc43 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:00:41.393384 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc005c66-452f-463c-90ab-a82f9f0ebc43 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:00:41.393660 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc005c66-452f-463c-90ab-a82f9f0ebc43 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:00:41.463862 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc005c66-452f-463c-90ab-a82f9f0ebc43 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:00:41.465245 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc005c66-452f-463c-90ab-a82f9f0ebc43 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:00:41.534313 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:00:41.573051 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 561/1128] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Jul 13 22:00:41 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 335 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 13 22:00:41.579513 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4941631d-e5ca-4263-9117-3113faf0f2ca None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:00:41.628248 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:00:41.692083 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4941631d-e5ca-4263-9117-3113faf0f2ca None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:00:41.727800 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1b88377-534e-476e-99ff-fca8797ee423 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:00:41.728590 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b88377-534e-476e-99ff-fca8797ee423 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:00:41.728864 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b88377-534e-476e-99ff-fca8797ee423 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:00:41.729132 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1b88377-534e-476e-99ff-fca8797ee423 None admin] PATH_INFO: `/v3/projects/3d8921c7315943f4a11901f37140ba1e` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:00:41.730426 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b88377-534e-476e-99ff-fca8797ee423 None admin] RBAC: Authorizing `identity:get_project(project_id=3d8921c7315943f4a11901f37140ba1e)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:00:41.731891 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1b88377-534e-476e-99ff-fca8797ee423 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:00:41.733571 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 568/1129] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Jul 13 22:00:41 2022] GET /identity/v3/projects/3d8921c7315943f4a11901f37140ba1e => generated 311 bytes in 155 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 22:00:41.740502 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc005c66-452f-463c-90ab-a82f9f0ebc43 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:00:41.854361 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc005c66-452f-463c-90ab-a82f9f0ebc43 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:00:41.892188 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-789ee936-95a4-4547-a151-21219a466062 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:00:41.892823 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789ee936-95a4-4547-a151-21219a466062 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:00:41.893092 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789ee936-95a4-4547-a151-21219a466062 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:00:41.893371 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-789ee936-95a4-4547-a151-21219a466062 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:00:41.894197 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789ee936-95a4-4547-a151-21219a466062 None admin] RBAC: Adding query filter params (role.id=db84c6c897534af1a60b9ce62532f383, scope.project.id=3d8921c7315943f4a11901f37140ba1e, user.id=5d882bf5681b4b15ba780ba16b0e7106) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 22:00:41.894526 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789ee936-95a4-4547-a151-21219a466062 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:00:41.895973 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-789ee936-95a4-4547-a151-21219a466062 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:00:41.902449 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 562/1130] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Jul 13 22:00:41 2022] GET /identity/v3/role_assignments?user.id=5d882bf5681b4b15ba780ba16b0e7106&scope.project.id=3d8921c7315943f4a11901f37140ba1e&role.id=db84c6c897534af1a60b9ce62532f383 => generated 608 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 22:00:43.911823 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e0f667-637c-4a84-b2fe-b47c664917fa None None] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:00:43.912370 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e0f667-637c-4a84-b2fe-b47c664917fa None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:00:43.912370 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37e0f667-637c-4a84-b2fe-b47c664917fa None None] PATH_INFO: `/` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:00:43.913778 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 569/1131] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jul 13 22:00:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 22:00:43.920269 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdd1306-ba22-4b1d-89a1-2529a89bcbd3 None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:00:43.921056 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdd1306-ba22-4b1d-89a1-2529a89bcbd3 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:00:43.921206 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cdd1306-ba22-4b1d-89a1-2529a89bcbd3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:00:43.943556 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-8cdd1306-ba22-4b1d-89a1-2529a89bcbd3 None None] MFA Rules not processed for user `cc99c93da5074cf5a48bc2aa9a92c5e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 22:00:43.944836 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-8cdd1306-ba22-4b1d-89a1-2529a89bcbd3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:00:43.974397 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:00:44.024869 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 563/1132] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jul 13 22:00:43 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 13 22:00:44.048582 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e0f667-637c-4a84-b2fe-b47c664917fa None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:00:44.152096 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37e0f667-637c-4a84-b2fe-b47c664917fa None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:00:44.165346 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:00:44.187706 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c63fd9fb-3a19-4f16-959f-ecdaf0740baf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:00:44.188501 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63fd9fb-3a19-4f16-959f-ecdaf0740baf None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:00:44.188779 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63fd9fb-3a19-4f16-959f-ecdaf0740baf None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:00:44.189051 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c63fd9fb-3a19-4f16-959f-ecdaf0740baf None admin] PATH_INFO: `/v3/domains/default` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:00:44.190399 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c63fd9fb-3a19-4f16-959f-ecdaf0740baf None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:00:44.191882 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c63fd9fb-3a19-4f16-959f-ecdaf0740baf None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:00:44.193496 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 570/1133] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Jul 13 22:00:44 2022] GET /identity/v3/domains/default => generated 199 bytes in 145 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 22:00:44.200041 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cdd1306-ba22-4b1d-89a1-2529a89bcbd3 None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:00:44.305458 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cdd1306-ba22-4b1d-89a1-2529a89bcbd3 None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:00:44.340551 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c519e97c-e577-4e2f-bb20-ee08fc751f4b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:00:44.341193 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c519e97c-e577-4e2f-bb20-ee08fc751f4b None admin] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:00:44.341467 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c519e97c-e577-4e2f-bb20-ee08fc751f4b None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:00:44.341734 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c519e97c-e577-4e2f-bb20-ee08fc751f4b None admin] PATH_INFO: `/v3/projects` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:00:44.342435 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c519e97c-e577-4e2f-bb20-ee08fc751f4b None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:00:44.343893 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c519e97c-e577-4e2f-bb20-ee08fc751f4b None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:00:44.379191 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 564/1134] 10.222.0.52 () {60 vars in 1141 bytes} [Wed Jul 13 22:00:44 2022] POST /identity/v3/projects => generated 311 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 22:00:46.401049 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10522a5-c111-4b59-8906-f7af5d2e5db6 None None] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:00:46.401964 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10522a5-c111-4b59-8906-f7af5d2e5db6 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:00:46.401964 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f10522a5-c111-4b59-8906-f7af5d2e5db6 None None] PATH_INFO: `/` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:00:46.404091 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 571/1135] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jul 13 22:00:46 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 22:00:46.409541 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95fc1dd-5995-4aec-93a5-fc72d6c01469 None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:00:46.410000 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95fc1dd-5995-4aec-93a5-fc72d6c01469 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:00:46.410098 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d95fc1dd-5995-4aec-93a5-fc72d6c01469 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:00:46.428199 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-d95fc1dd-5995-4aec-93a5-fc72d6c01469 None None] MFA Rules not processed for user `cc99c93da5074cf5a48bc2aa9a92c5e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 22:00:46.429404 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-d95fc1dd-5995-4aec-93a5-fc72d6c01469 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:00:46.438565 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:00:46.501202 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 565/1136] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jul 13 22:00:46 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 13 22:00:46.526371 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10522a5-c111-4b59-8906-f7af5d2e5db6 None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:00:46.637432 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f10522a5-c111-4b59-8906-f7af5d2e5db6 None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:00:46.673231 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7772f8ba-3b8b-4eaa-bf25-bb008a10e59e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:00:46.673958 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7772f8ba-3b8b-4eaa-bf25-bb008a10e59e None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:00:46.674234 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7772f8ba-3b8b-4eaa-bf25-bb008a10e59e None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:00:46.674536 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7772f8ba-3b8b-4eaa-bf25-bb008a10e59e None admin] PATH_INFO: `/v3/domains/default` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:00:46.675825 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7772f8ba-3b8b-4eaa-bf25-bb008a10e59e None admin] RBAC: Authorizing `identity:get_domain(domain_id=default)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:00:46.677262 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7772f8ba-3b8b-4eaa-bf25-bb008a10e59e None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:00:46.678965 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 572/1137] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Jul 13 22:00:46 2022] GET /identity/v3/domains/default => generated 199 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 22:00:46.688108 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95fc1dd-5995-4aec-93a5-fc72d6c01469 None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:00:46.792233 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d95fc1dd-5995-4aec-93a5-fc72d6c01469 None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:00:46.830734 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56359c92-267d-422f-a183-e2aa3e67b127 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:00:46.831396 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56359c92-267d-422f-a183-e2aa3e67b127 None admin] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:00:46.831676 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56359c92-267d-422f-a183-e2aa3e67b127 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:00:46.831952 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56359c92-267d-422f-a183-e2aa3e67b127 None admin] PATH_INFO: `/v3/users` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:00:46.832661 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56359c92-267d-422f-a183-e2aa3e67b127 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:00:46.834093 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56359c92-267d-422f-a183-e2aa3e67b127 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:00:46.852012 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:00:46.857519 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 566/1138] 10.222.0.52 () {60 vars in 1132 bytes} [Wed Jul 13 22:00:46 2022] POST /identity/v3/users => generated 286 bytes in 170 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 22:00:48.892023 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e20167-db18-44a5-a7b2-3729108e23d1 None None] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:00:48.892848 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e20167-db18-44a5-a7b2-3729108e23d1 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:00:48.892848 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2e20167-db18-44a5-a7b2-3729108e23d1 None None] PATH_INFO: `/` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:00:48.894976 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 573/1139] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jul 13 22:00:48 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 22:00:48.900658 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910b61fe-4fe1-417b-8c8d-7789358aeb88 None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:00:48.901102 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910b61fe-4fe1-417b-8c8d-7789358aeb88 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:00:48.901202 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-910b61fe-4fe1-417b-8c8d-7789358aeb88 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:00:48.919280 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-910b61fe-4fe1-417b-8c8d-7789358aeb88 None None] MFA Rules not processed for user `cc99c93da5074cf5a48bc2aa9a92c5e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 22:00:48.920445 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-910b61fe-4fe1-417b-8c8d-7789358aeb88 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:00:48.999479 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 567/1140] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jul 13 22:00:48 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 13 22:00:49.025902 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e20167-db18-44a5-a7b2-3729108e23d1 None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:00:49.082064 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:00:49.146324 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2e20167-db18-44a5-a7b2-3729108e23d1 None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:00:49.181279 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1be8177-b668-47c0-bf32-4d3e9f4a5051 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:00:49.182062 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1be8177-b668-47c0-bf32-4d3e9f4a5051 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:00:49.182340 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1be8177-b668-47c0-bf32-4d3e9f4a5051 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:00:49.182646 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1be8177-b668-47c0-bf32-4d3e9f4a5051 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:00:49.193695 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1be8177-b668-47c0-bf32-4d3e9f4a5051 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 22:00:49.201591 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application [None req-c1be8177-b668-47c0-bf32-4d3e9f4a5051 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 13 22:00:49.201591 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 22:00:49.201591 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 22:00:49.201591 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 22:00:49.201591 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 22:00:49.201591 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 22:00:49.201591 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 22:00:49.201591 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 22:00:49.201591 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 22:00:49.201591 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 22:00:49.201591 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 22:00:49.201591 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 22:00:49.201591 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 22:00:49.201591 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 22:00:49.201591 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 22:00:49.201591 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 22:00:49.202750 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 22:00:49.202750 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 22:00:49.202750 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 22:00:49.202750 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 22:00:49.202750 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 22:00:49.202750 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 22:00:49.202750 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 22:00:49.202750 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 22:00:49.202750 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 22:00:49.202750 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 22:00:49.202750 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 22:00:49.202750 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application with Lock( Jul 13 22:00:49.202750 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 22:00:49.202750 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._enter() Jul 13 22:00:49.202750 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 22:00:49.202750 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 22:00:49.204019 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 22:00:49.204019 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.creator() Jul 13 22:00:49.204019 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 22:00:49.204019 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application created_value = creator( Jul 13 22:00:49.204019 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 22:00:49.204019 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 22:00:49.204019 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 22:00:49.204019 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 22:00:49.204019 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 22:00:49.204019 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 22:00:49.204019 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 13 22:00:49.204019 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Jul 13 22:00:49.204019 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 574/1141] 10.222.0.52 () {56 vars in 1096 bytes} [Wed Jul 13 22:00:49 2022] GET /identity/v3/roles/admin => generated 83 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 22:00:49.210828 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910b61fe-4fe1-417b-8c8d-7789358aeb88 None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:00:49.311349 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-910b61fe-4fe1-417b-8c8d-7789358aeb88 None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:00:49.340760 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:00:49.347417 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c24d4b-e48b-4db7-8ed1-0fbff4f98a41 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:00:49.348085 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c24d4b-e48b-4db7-8ed1-0fbff4f98a41 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:00:49.348359 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c24d4b-e48b-4db7-8ed1-0fbff4f98a41 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:00:49.348631 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46c24d4b-e48b-4db7-8ed1-0fbff4f98a41 None admin] PATH_INFO: `/v3/roles` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:00:49.349381 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c24d4b-e48b-4db7-8ed1-0fbff4f98a41 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 22:00:49.349685 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c24d4b-e48b-4db7-8ed1-0fbff4f98a41 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:00:49.351126 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46c24d4b-e48b-4db7-8ed1-0fbff4f98a41 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:00:49.359391 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 568/1142] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Jul 13 22:00:49 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 22:00:49.366147 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1be8177-b668-47c0-bf32-4d3e9f4a5051 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:00:49.478357 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1be8177-b668-47c0-bf32-4d3e9f4a5051 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:00:49.487073 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:00:49.513008 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a345673-5441-4248-9285-cfb47f8e25f1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:00:49.513873 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a345673-5441-4248-9285-cfb47f8e25f1 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:00:49.514144 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a345673-5441-4248-9285-cfb47f8e25f1 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:00:49.514417 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a345673-5441-4248-9285-cfb47f8e25f1 None admin] PATH_INFO: `/v3/users/787e7a19a6d84647a489379900aa75cd` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:00:49.527996 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a345673-5441-4248-9285-cfb47f8e25f1 None admin] RBAC: Authorizing `identity:get_user(user_id=787e7a19a6d84647a489379900aa75cd)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:00:49.529441 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a345673-5441-4248-9285-cfb47f8e25f1 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:00:49.531111 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 575/1143] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Jul 13 22:00:49 2022] GET /identity/v3/users/787e7a19a6d84647a489379900aa75cd => generated 286 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 22:00:49.536973 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c24d4b-e48b-4db7-8ed1-0fbff4f98a41 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:00:49.640602 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46c24d4b-e48b-4db7-8ed1-0fbff4f98a41 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:00:49.678089 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e24951-5056-4124-99e9-179f26ade4cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:00:49.678769 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e24951-5056-4124-99e9-179f26ade4cd None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:00:49.679218 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e24951-5056-4124-99e9-179f26ade4cd None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:00:49.679882 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e24951-5056-4124-99e9-179f26ade4cd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:00:49.749726 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:00:49.760349 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e24951-5056-4124-99e9-179f26ade4cd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:00:49.761805 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e24951-5056-4124-99e9-179f26ade4cd None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:00:49.863507 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 569/1144] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Jul 13 22:00:49 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 327 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 13 22:00:49.870472 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a345673-5441-4248-9285-cfb47f8e25f1 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:00:49.985306 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a345673-5441-4248-9285-cfb47f8e25f1 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:00:50.020903 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a9f102b-943a-4f1d-8dd3-86ba81325fb4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:00:50.021695 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9f102b-943a-4f1d-8dd3-86ba81325fb4 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:00:50.021975 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9f102b-943a-4f1d-8dd3-86ba81325fb4 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:00:50.022253 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a9f102b-943a-4f1d-8dd3-86ba81325fb4 None admin] PATH_INFO: `/v3/projects/653d534124e5423583669c3e1b78fb33` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:00:50.023692 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9f102b-943a-4f1d-8dd3-86ba81325fb4 None admin] RBAC: Authorizing `identity:get_project(project_id=653d534124e5423583669c3e1b78fb33)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:00:50.025142 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a9f102b-943a-4f1d-8dd3-86ba81325fb4 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:00:50.026884 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 576/1145] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Jul 13 22:00:49 2022] GET /identity/v3/projects/653d534124e5423583669c3e1b78fb33 => generated 311 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 22:00:50.033698 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e24951-5056-4124-99e9-179f26ade4cd None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:00:50.144251 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:00:50.153610 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e24951-5056-4124-99e9-179f26ade4cd None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:00:50.188014 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a022f0-a05c-4dd6-9199-5f94d3afbc00 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:00:50.188657 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a022f0-a05c-4dd6-9199-5f94d3afbc00 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:00:50.188931 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a022f0-a05c-4dd6-9199-5f94d3afbc00 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:00:50.189198 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a022f0-a05c-4dd6-9199-5f94d3afbc00 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:00:50.190021 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a022f0-a05c-4dd6-9199-5f94d3afbc00 None admin] RBAC: Adding query filter params (role.id=44efba2579f342df8a0789c59d163ec3, scope.project.id=653d534124e5423583669c3e1b78fb33, user.id=787e7a19a6d84647a489379900aa75cd) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 22:00:50.190329 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a022f0-a05c-4dd6-9199-5f94d3afbc00 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:00:50.191792 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a022f0-a05c-4dd6-9199-5f94d3afbc00 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:00:50.197521 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 570/1146] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Jul 13 22:00:50 2022] GET /identity/v3/role_assignments?user.id=787e7a19a6d84647a489379900aa75cd&scope.project.id=653d534124e5423583669c3e1b78fb33&role.id=44efba2579f342df8a0789c59d163ec3 => generated 259 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 22:00:52.183616 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092a6739-8e82-4f36-9876-7ccd2ebfab34 None None] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:00:52.184527 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092a6739-8e82-4f36-9876-7ccd2ebfab34 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:00:52.184527 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-092a6739-8e82-4f36-9876-7ccd2ebfab34 None None] PATH_INFO: `/` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:00:52.186752 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 577/1147] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jul 13 22:00:52 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 22:00:52.193083 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acec7bdd-7ea5-44b9-ba4e-7ba3a7d81c98 None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:00:52.193813 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acec7bdd-7ea5-44b9-ba4e-7ba3a7d81c98 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:00:52.194001 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acec7bdd-7ea5-44b9-ba4e-7ba3a7d81c98 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:00:52.217797 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-acec7bdd-7ea5-44b9-ba4e-7ba3a7d81c98 None None] MFA Rules not processed for user `cc99c93da5074cf5a48bc2aa9a92c5e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 22:00:52.219061 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-acec7bdd-7ea5-44b9-ba4e-7ba3a7d81c98 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:00:52.297484 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 571/1148] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jul 13 22:00:52 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 13 22:00:52.320695 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092a6739-8e82-4f36-9876-7ccd2ebfab34 None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:00:52.360991 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:00:52.431091 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-092a6739-8e82-4f36-9876-7ccd2ebfab34 None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:00:52.466427 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88e652b-40dc-46d2-84b0-0acb0ca378bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:00:52.467176 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88e652b-40dc-46d2-84b0-0acb0ca378bd None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:00:52.467454 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88e652b-40dc-46d2-84b0-0acb0ca378bd None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:00:52.467723 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d88e652b-40dc-46d2-84b0-0acb0ca378bd None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:00:52.478752 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d88e652b-40dc-46d2-84b0-0acb0ca378bd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 22:00:52.486285 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application [None req-d88e652b-40dc-46d2-84b0-0acb0ca378bd None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 13 22:00:52.486285 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 22:00:52.486285 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 22:00:52.486285 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 22:00:52.486285 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 22:00:52.486285 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 22:00:52.486285 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 22:00:52.486285 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 22:00:52.486285 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 22:00:52.486285 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 22:00:52.486285 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 22:00:52.486285 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 22:00:52.486285 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 22:00:52.486285 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 22:00:52.486285 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 22:00:52.486285 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 22:00:52.487418 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 22:00:52.487418 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 22:00:52.487418 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 22:00:52.487418 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 22:00:52.487418 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 22:00:52.487418 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 22:00:52.487418 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 22:00:52.487418 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 22:00:52.487418 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 22:00:52.487418 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 22:00:52.487418 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 22:00:52.487418 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application with Lock( Jul 13 22:00:52.487418 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 22:00:52.487418 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._enter() Jul 13 22:00:52.487418 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 22:00:52.487418 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 22:00:52.488650 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 22:00:52.488650 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.creator() Jul 13 22:00:52.488650 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 22:00:52.488650 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application created_value = creator( Jul 13 22:00:52.488650 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 22:00:52.488650 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 22:00:52.488650 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 22:00:52.488650 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 22:00:52.488650 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 22:00:52.488650 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 22:00:52.488650 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 13 22:00:52.488650 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Jul 13 22:00:52.488650 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 578/1149] 10.222.0.52 () {56 vars in 1096 bytes} [Wed Jul 13 22:00:52 2022] GET /identity/v3/roles/admin => generated 83 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 22:00:52.496777 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acec7bdd-7ea5-44b9-ba4e-7ba3a7d81c98 None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:00:52.590977 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:00:52.606978 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acec7bdd-7ea5-44b9-ba4e-7ba3a7d81c98 None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:00:52.643103 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218a4310-31c8-48aa-ab1b-5f90bd8c7624 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:00:52.643766 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218a4310-31c8-48aa-ab1b-5f90bd8c7624 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:00:52.644045 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218a4310-31c8-48aa-ab1b-5f90bd8c7624 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:00:52.644323 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-218a4310-31c8-48aa-ab1b-5f90bd8c7624 None admin] PATH_INFO: `/v3/roles` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:00:52.645086 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218a4310-31c8-48aa-ab1b-5f90bd8c7624 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 22:00:52.645368 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218a4310-31c8-48aa-ab1b-5f90bd8c7624 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:00:52.646812 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-218a4310-31c8-48aa-ab1b-5f90bd8c7624 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:00:52.655048 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 572/1150] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Jul 13 22:00:52 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 22:00:52.661173 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88e652b-40dc-46d2-84b0-0acb0ca378bd None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:00:52.763469 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:00:52.770821 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d88e652b-40dc-46d2-84b0-0acb0ca378bd None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:00:52.805435 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70034e59-89fa-4ac0-ba47-77f43e9fa34c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:00:52.806299 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70034e59-89fa-4ac0-ba47-77f43e9fa34c None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:00:52.806612 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70034e59-89fa-4ac0-ba47-77f43e9fa34c None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:00:52.806890 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70034e59-89fa-4ac0-ba47-77f43e9fa34c None admin] PATH_INFO: `/v3/users/787e7a19a6d84647a489379900aa75cd` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:00:52.808129 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70034e59-89fa-4ac0-ba47-77f43e9fa34c None admin] RBAC: Authorizing `identity:get_user(user_id=787e7a19a6d84647a489379900aa75cd)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:00:52.809581 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-70034e59-89fa-4ac0-ba47-77f43e9fa34c None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:00:52.811237 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 579/1151] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Jul 13 22:00:52 2022] GET /identity/v3/users/787e7a19a6d84647a489379900aa75cd => generated 286 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 22:00:52.817890 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218a4310-31c8-48aa-ab1b-5f90bd8c7624 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:00:52.928411 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-218a4310-31c8-48aa-ab1b-5f90bd8c7624 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:00:52.964374 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b734f91e-17fc-40fb-be68-c68ceadad3d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:00:52.965230 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b734f91e-17fc-40fb-be68-c68ceadad3d9 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:00:52.965504 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b734f91e-17fc-40fb-be68-c68ceadad3d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:00:52.965778 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b734f91e-17fc-40fb-be68-c68ceadad3d9 None admin] PATH_INFO: `/v3/projects/653d534124e5423583669c3e1b78fb33` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:00:52.967106 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b734f91e-17fc-40fb-be68-c68ceadad3d9 None admin] RBAC: Authorizing `identity:get_project(project_id=653d534124e5423583669c3e1b78fb33)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:00:52.968557 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b734f91e-17fc-40fb-be68-c68ceadad3d9 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:00:52.970276 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 573/1152] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Jul 13 22:00:52 2022] GET /identity/v3/projects/653d534124e5423583669c3e1b78fb33 => generated 311 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 22:00:52.976574 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70034e59-89fa-4ac0-ba47-77f43e9fa34c None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:00:53.081020 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-70034e59-89fa-4ac0-ba47-77f43e9fa34c None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:00:53.115904 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b5af5d8-a66d-41c7-afff-e4e922868b19 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:00:53.116829 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5af5d8-a66d-41c7-afff-e4e922868b19 None admin] REQUEST_METHOD: `PUT` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:00:53.117135 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5af5d8-a66d-41c7-afff-e4e922868b19 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:00:53.117409 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b5af5d8-a66d-41c7-afff-e4e922868b19 None admin] PATH_INFO: `/v3/projects/653d534124e5423583669c3e1b78fb33/users/787e7a19a6d84647a489379900aa75cd/roles/44efba2579f342df8a0789c59d163ec3` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:00:53.119508 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b5af5d8-a66d-41c7-afff-e4e922868b19 None admin] RBAC: Authorizing `identity:create_grant(project_id=653d534124e5423583669c3e1b78fb33, user_id=787e7a19a6d84647a489379900aa75cd, role_id=44efba2579f342df8a0789c59d163ec3)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:00:53.120955 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b5af5d8-a66d-41c7-afff-e4e922868b19 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:00:53.137283 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 580/1153] 10.222.0.52 () {58 vars in 1439 bytes} [Wed Jul 13 22:00:52 2022] PUT /identity/v3/projects/653d534124e5423583669c3e1b78fb33/users/787e7a19a6d84647a489379900aa75cd/roles/44efba2579f342df8a0789c59d163ec3 => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 22:00:55.110826 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317f0112-2b8e-48fc-a785-c5e1690b75ef None None] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:00:55.111402 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317f0112-2b8e-48fc-a785-c5e1690b75ef None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:00:55.111402 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317f0112-2b8e-48fc-a785-c5e1690b75ef None None] PATH_INFO: `/` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:00:55.112757 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 574/1154] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jul 13 22:00:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 22:00:55.119273 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7468ab88-693c-4dcd-91a6-8a222761d012 None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:00:55.119966 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7468ab88-693c-4dcd-91a6-8a222761d012 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:00:55.120158 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7468ab88-693c-4dcd-91a6-8a222761d012 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:00:55.148572 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-7468ab88-693c-4dcd-91a6-8a222761d012 None None] MFA Rules not processed for user `cc99c93da5074cf5a48bc2aa9a92c5e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 22:00:55.149771 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-7468ab88-693c-4dcd-91a6-8a222761d012 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:00:55.168012 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:00:55.240939 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 581/1155] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jul 13 22:00:55 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 13 22:00:55.277610 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317f0112-2b8e-48fc-a785-c5e1690b75ef None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:00:55.302459 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:00:55.390807 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317f0112-2b8e-48fc-a785-c5e1690b75ef None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:00:55.426190 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b3f925-70a7-4ad2-ae61-30d31a2cef54 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:00:55.427006 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b3f925-70a7-4ad2-ae61-30d31a2cef54 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:00:55.427288 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b3f925-70a7-4ad2-ae61-30d31a2cef54 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:00:55.427564 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95b3f925-70a7-4ad2-ae61-30d31a2cef54 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:00:55.438887 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95b3f925-70a7-4ad2-ae61-30d31a2cef54 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 22:00:55.446726 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application [None req-95b3f925-70a7-4ad2-ae61-30d31a2cef54 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 13 22:00:55.446726 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 22:00:55.446726 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 22:00:55.446726 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 22:00:55.446726 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 22:00:55.446726 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 22:00:55.446726 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 22:00:55.446726 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 22:00:55.446726 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 22:00:55.446726 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 22:00:55.446726 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 22:00:55.446726 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 22:00:55.446726 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 22:00:55.446726 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 22:00:55.446726 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 22:00:55.446726 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 22:00:55.447805 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 22:00:55.447805 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 22:00:55.447805 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 22:00:55.447805 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 22:00:55.447805 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 22:00:55.447805 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 22:00:55.447805 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 22:00:55.447805 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 22:00:55.447805 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 22:00:55.447805 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 22:00:55.447805 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 22:00:55.447805 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application with Lock( Jul 13 22:00:55.447805 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 22:00:55.447805 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._enter() Jul 13 22:00:55.447805 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 22:00:55.447805 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 22:00:55.449187 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 22:00:55.449187 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.creator() Jul 13 22:00:55.449187 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 22:00:55.449187 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application created_value = creator( Jul 13 22:00:55.449187 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 22:00:55.449187 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 22:00:55.449187 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 22:00:55.449187 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 22:00:55.449187 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 22:00:55.449187 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 22:00:55.449187 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 13 22:00:55.449187 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Jul 13 22:00:55.449187 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 575/1156] 10.222.0.52 () {56 vars in 1096 bytes} [Wed Jul 13 22:00:55 2022] GET /identity/v3/roles/admin => generated 83 bytes in 171 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 22:00:55.455230 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7468ab88-693c-4dcd-91a6-8a222761d012 None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:00:55.575732 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7468ab88-693c-4dcd-91a6-8a222761d012 None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:00:55.610649 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893363e2-de07-49e5-bcd3-a61717c581c3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:00:55.611315 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893363e2-de07-49e5-bcd3-a61717c581c3 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:00:55.611594 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893363e2-de07-49e5-bcd3-a61717c581c3 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:00:55.611877 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-893363e2-de07-49e5-bcd3-a61717c581c3 None admin] PATH_INFO: `/v3/roles` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:00:55.612630 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893363e2-de07-49e5-bcd3-a61717c581c3 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 22:00:55.612947 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893363e2-de07-49e5-bcd3-a61717c581c3 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:00:55.614335 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-893363e2-de07-49e5-bcd3-a61717c581c3 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:00:55.622691 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 582/1157] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Jul 13 22:00:55 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 22:00:55.629385 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b3f925-70a7-4ad2-ae61-30d31a2cef54 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:00:55.709647 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:00:55.733856 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95b3f925-70a7-4ad2-ae61-30d31a2cef54 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:00:55.770811 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6216c564-85ca-47b6-a61b-4b5551e8ea56 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:00:55.771680 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6216c564-85ca-47b6-a61b-4b5551e8ea56 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:00:55.771960 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6216c564-85ca-47b6-a61b-4b5551e8ea56 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:00:55.772230 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6216c564-85ca-47b6-a61b-4b5551e8ea56 None admin] PATH_INFO: `/v3/users/787e7a19a6d84647a489379900aa75cd` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:00:55.773633 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6216c564-85ca-47b6-a61b-4b5551e8ea56 None admin] RBAC: Authorizing `identity:get_user(user_id=787e7a19a6d84647a489379900aa75cd)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:00:55.775123 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6216c564-85ca-47b6-a61b-4b5551e8ea56 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:00:55.776778 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 576/1158] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Jul 13 22:00:55 2022] GET /identity/v3/users/787e7a19a6d84647a489379900aa75cd => generated 286 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 22:00:55.785805 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893363e2-de07-49e5-bcd3-a61717c581c3 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:00:55.803295 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:00:55.888303 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-893363e2-de07-49e5-bcd3-a61717c581c3 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:00:55.922290 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd4e4a0-7bc6-43d6-ac95-3bf00e667b66 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:00:55.922968 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd4e4a0-7bc6-43d6-ac95-3bf00e667b66 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:00:55.923251 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd4e4a0-7bc6-43d6-ac95-3bf00e667b66 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:00:55.923525 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd4e4a0-7bc6-43d6-ac95-3bf00e667b66 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:00:55.991735 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd4e4a0-7bc6-43d6-ac95-3bf00e667b66 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:00:55.993112 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd4e4a0-7bc6-43d6-ac95-3bf00e667b66 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:00:56.007484 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:00:56.093672 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 583/1159] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Jul 13 22:00:55 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 308 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 13 22:00:56.111029 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6216c564-85ca-47b6-a61b-4b5551e8ea56 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:00:56.219566 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6216c564-85ca-47b6-a61b-4b5551e8ea56 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:00:56.254739 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e24f7bb-2900-48d3-b2bd-7dc5940559a6 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:00:56.255526 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e24f7bb-2900-48d3-b2bd-7dc5940559a6 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:00:56.255800 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e24f7bb-2900-48d3-b2bd-7dc5940559a6 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:00:56.256075 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e24f7bb-2900-48d3-b2bd-7dc5940559a6 None admin] PATH_INFO: `/v3/projects/653d534124e5423583669c3e1b78fb33` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:00:56.257358 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e24f7bb-2900-48d3-b2bd-7dc5940559a6 None admin] RBAC: Authorizing `identity:get_project(project_id=653d534124e5423583669c3e1b78fb33)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:00:56.258821 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e24f7bb-2900-48d3-b2bd-7dc5940559a6 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:00:56.260518 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 577/1160] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Jul 13 22:00:56 2022] GET /identity/v3/projects/653d534124e5423583669c3e1b78fb33 => generated 311 bytes in 150 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 22:00:56.270093 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd4e4a0-7bc6-43d6-ac95-3bf00e667b66 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:00:56.374148 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd4e4a0-7bc6-43d6-ac95-3bf00e667b66 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:00:56.389106 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:00:56.411246 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02b1295d-2fa7-483c-b8a7-5fc099fcf498 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:00:56.411894 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b1295d-2fa7-483c-b8a7-5fc099fcf498 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:00:56.412170 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b1295d-2fa7-483c-b8a7-5fc099fcf498 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:00:56.412438 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02b1295d-2fa7-483c-b8a7-5fc099fcf498 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:00:56.413595 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b1295d-2fa7-483c-b8a7-5fc099fcf498 None admin] RBAC: Adding query filter params (role.id=44efba2579f342df8a0789c59d163ec3, scope.project.id=653d534124e5423583669c3e1b78fb33, user.id=787e7a19a6d84647a489379900aa75cd) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 22:00:56.414070 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b1295d-2fa7-483c-b8a7-5fc099fcf498 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:00:56.415799 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02b1295d-2fa7-483c-b8a7-5fc099fcf498 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:00:56.421773 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 584/1161] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Jul 13 22:00:56 2022] GET /identity/v3/role_assignments?user.id=787e7a19a6d84647a489379900aa75cd&scope.project.id=653d534124e5423583669c3e1b78fb33&role.id=44efba2579f342df8a0789c59d163ec3 => generated 608 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 22:00:58.419516 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82202087-e86d-4e1f-913b-c485d370a1c4 None None] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:00:58.420122 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82202087-e86d-4e1f-913b-c485d370a1c4 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:00:58.420122 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82202087-e86d-4e1f-913b-c485d370a1c4 None None] PATH_INFO: `/` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:00:58.421439 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 578/1162] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jul 13 22:00:58 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 22:00:58.427895 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dab68e6-4d4c-40fc-85ab-c7c0e5a5dc7e None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:00:58.428590 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dab68e6-4d4c-40fc-85ab-c7c0e5a5dc7e None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:00:58.428830 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dab68e6-4d4c-40fc-85ab-c7c0e5a5dc7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:00:58.452511 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-4dab68e6-4d4c-40fc-85ab-c7c0e5a5dc7e None None] MFA Rules not processed for user `cc99c93da5074cf5a48bc2aa9a92c5e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 22:00:58.453715 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-4dab68e6-4d4c-40fc-85ab-c7c0e5a5dc7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:00:58.523738 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 585/1163] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jul 13 22:00:58 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 13 22:00:58.546165 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82202087-e86d-4e1f-913b-c485d370a1c4 None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:00:58.563753 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:00:58.656182 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82202087-e86d-4e1f-913b-c485d370a1c4 None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:00:58.692297 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60eccc2-0140-45ad-9a3b-e2a62fae71db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:00:58.692871 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60eccc2-0140-45ad-9a3b-e2a62fae71db None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:00:58.693151 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60eccc2-0140-45ad-9a3b-e2a62fae71db None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:00:58.693421 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b60eccc2-0140-45ad-9a3b-e2a62fae71db None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:00:58.768460 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60eccc2-0140-45ad-9a3b-e2a62fae71db None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:00:58.769905 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b60eccc2-0140-45ad-9a3b-e2a62fae71db None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:00:58.785461 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:00:58.874944 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 579/1164] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Jul 13 22:00:58 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 13 22:00:58.881323 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dab68e6-4d4c-40fc-85ab-c7c0e5a5dc7e None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:00:58.986281 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dab68e6-4d4c-40fc-85ab-c7c0e5a5dc7e None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:00:58.996019 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:00:59.023711 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e15d7779-c013-43a4-89bb-a23d63cb7094 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:00:59.024520 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15d7779-c013-43a4-89bb-a23d63cb7094 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:00:59.024799 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15d7779-c013-43a4-89bb-a23d63cb7094 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:00:59.025070 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e15d7779-c013-43a4-89bb-a23d63cb7094 None admin] PATH_INFO: `/v3/domains/swift_test` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:00:59.032074 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15d7779-c013-43a4-89bb-a23d63cb7094 None admin] RBAC: Authorizing `identity:get_domain(domain_id=swift_test)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:00:59.033527 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e15d7779-c013-43a4-89bb-a23d63cb7094 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:00:59.047904 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application [None req-e15d7779-c013-43a4-89bb-a23d63cb7094 None admin] Could not find domain: swift_test.: keystone.exception.DomainNotFound: Could not find domain: swift_test. Jul 13 22:00:59.047904 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 22:00:59.047904 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 712, in get_domain Jul 13 22:00:59.047904 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application project = self.driver.get_project(domain_id) Jul 13 22:00:59.047904 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 13 22:00:59.047904 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 13 22:00:59.047904 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 13 22:00:59.047904 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 13 22:00:59.047904 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: swift_test. Jul 13 22:00:59.047904 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Jul 13 22:00:59.047904 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 13 22:00:59.047904 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Jul 13 22:00:59.047904 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 22:00:59.047904 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 22:00:59.047904 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 22:00:59.047904 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 22:00:59.047904 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 22:00:59.049081 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 22:00:59.049081 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 22:00:59.049081 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 22:00:59.049081 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 22:00:59.049081 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 22:00:59.049081 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 22:00:59.049081 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 22:00:59.049081 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 22:00:59.049081 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 22:00:59.049081 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 22:00:59.049081 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 22:00:59.049081 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 22:00:59.049081 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 22:00:59.049081 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 22:00:59.049081 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 22:00:59.049081 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 22:00:59.050087 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 22:00:59.050087 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application with Lock( Jul 13 22:00:59.050087 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 22:00:59.050087 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._enter() Jul 13 22:00:59.050087 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 22:00:59.050087 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 22:00:59.050087 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 22:00:59.050087 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.creator() Jul 13 22:00:59.050087 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 22:00:59.050087 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application created_value = creator( Jul 13 22:00:59.050087 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 718, in get_domain Jul 13 22:00:59.050087 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 22:00:59.050087 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: swift_test. Jul 13 22:00:59.050087 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Jul 13 22:00:59.054179 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 586/1165] 10.222.0.52 () {56 vars in 1117 bytes} [Wed Jul 13 22:00:58 2022] GET /identity/v3/domains/swift_test => generated 90 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 22:00:59.055978 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60eccc2-0140-45ad-9a3b-e2a62fae71db None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:00:59.166916 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b60eccc2-0140-45ad-9a3b-e2a62fae71db None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:00:59.180084 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:00:59.203569 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39f49490-6eb0-4567-9937-b5ea745d8488 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:00:59.204224 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f49490-6eb0-4567-9937-b5ea745d8488 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:00:59.204512 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f49490-6eb0-4567-9937-b5ea745d8488 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:00:59.204784 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39f49490-6eb0-4567-9937-b5ea745d8488 None admin] PATH_INFO: `/v3/domains` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:00:59.205507 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f49490-6eb0-4567-9937-b5ea745d8488 None admin] RBAC: Adding query filter params (name=swift_test) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 22:00:59.205813 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f49490-6eb0-4567-9937-b5ea745d8488 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:00:59.207215 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39f49490-6eb0-4567-9937-b5ea745d8488 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:00:59.214697 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 580/1166] 10.222.0.52 () {56 vars in 1115 bytes} [Wed Jul 13 22:00:59 2022] GET /identity/v3/domains?name=swift_test => generated 125 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 22:01:01.212311 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471ed4ad-eef9-4ede-bfcc-9367800cb5f8 None None] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:01:01.213204 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471ed4ad-eef9-4ede-bfcc-9367800cb5f8 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:01:01.213204 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-471ed4ad-eef9-4ede-bfcc-9367800cb5f8 None None] PATH_INFO: `/` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:01:01.215440 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 587/1167] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jul 13 22:01:01 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 22:01:01.224099 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb888d7-8e89-42ea-8ff3-c87428ed3d76 None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:01:01.224518 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb888d7-8e89-42ea-8ff3-c87428ed3d76 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:01:01.224702 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb888d7-8e89-42ea-8ff3-c87428ed3d76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:01:01.250524 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-acb888d7-8e89-42ea-8ff3-c87428ed3d76 None None] MFA Rules not processed for user `cc99c93da5074cf5a48bc2aa9a92c5e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 22:01:01.251739 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-acb888d7-8e89-42ea-8ff3-c87428ed3d76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:01:01.336608 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 581/1168] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jul 13 22:01:01 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 13 22:01:01.359800 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471ed4ad-eef9-4ede-bfcc-9367800cb5f8 None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:01:01.479054 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-471ed4ad-eef9-4ede-bfcc-9367800cb5f8 None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:01:01.518523 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc97ac3a-c809-4fa6-ad69-82fb262ddca0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:01:01.519195 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc97ac3a-c809-4fa6-ad69-82fb262ddca0 None admin] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:01:01.519483 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc97ac3a-c809-4fa6-ad69-82fb262ddca0 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:01:01.519752 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc97ac3a-c809-4fa6-ad69-82fb262ddca0 None admin] PATH_INFO: `/v3/domains` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:01:01.520495 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc97ac3a-c809-4fa6-ad69-82fb262ddca0 None admin] RBAC: Authorizing `identity:create_domain()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:01:01.521880 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc97ac3a-c809-4fa6-ad69-82fb262ddca0 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:01:01.539810 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 588/1169] 10.222.0.52 () {60 vars in 1138 bytes} [Wed Jul 13 22:01:01 2022] POST /identity/v3/domains => generated 267 bytes in 180 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 22:01:03.581416 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8532f614-4c72-4cc4-b1c1-87476f7a2927 None None] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:01:03.581900 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8532f614-4c72-4cc4-b1c1-87476f7a2927 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:01:03.581990 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8532f614-4c72-4cc4-b1c1-87476f7a2927 None None] PATH_INFO: `/` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:01:03.583389 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 582/1170] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jul 13 22:01:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 22:01:03.589871 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863fd5a6-1bae-4e09-8558-2caae59c9a12 None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:01:03.590589 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863fd5a6-1bae-4e09-8558-2caae59c9a12 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:01:03.590855 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-863fd5a6-1bae-4e09-8558-2caae59c9a12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:01:03.618792 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-863fd5a6-1bae-4e09-8558-2caae59c9a12 None None] MFA Rules not processed for user `cc99c93da5074cf5a48bc2aa9a92c5e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 22:01:03.619998 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-863fd5a6-1bae-4e09-8558-2caae59c9a12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:01:03.643980 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:01:03.692282 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 589/1171] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jul 13 22:01:03 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 13 22:01:03.716746 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8532f614-4c72-4cc4-b1c1-87476f7a2927 None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:01:03.827815 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8532f614-4c72-4cc4-b1c1-87476f7a2927 None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:01:03.834852 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:01:03.863172 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-532101d5-a309-422f-8f58-1c53672dd307 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:01:03.864029 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532101d5-a309-422f-8f58-1c53672dd307 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:01:03.864299 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532101d5-a309-422f-8f58-1c53672dd307 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:01:03.864559 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-532101d5-a309-422f-8f58-1c53672dd307 None admin] PATH_INFO: `/v3/domains/2bb7deb7d8c140bb85a5e72f18ec3a10` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:01:03.875467 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532101d5-a309-422f-8f58-1c53672dd307 None admin] RBAC: Authorizing `identity:get_domain(domain_id=2bb7deb7d8c140bb85a5e72f18ec3a10)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:01:03.876912 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-532101d5-a309-422f-8f58-1c53672dd307 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:01:03.878570 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 583/1172] 10.222.0.52 () {56 vars in 1183 bytes} [Wed Jul 13 22:01:03 2022] GET /identity/v3/domains/2bb7deb7d8c140bb85a5e72f18ec3a10 => generated 267 bytes in 162 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 22:01:03.885617 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863fd5a6-1bae-4e09-8558-2caae59c9a12 None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:01:03.998712 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-863fd5a6-1bae-4e09-8558-2caae59c9a12 None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:01:04.034391 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0becf91-cc7b-4466-94ac-f4dcfbb1d575 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:01:04.035082 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0becf91-cc7b-4466-94ac-f4dcfbb1d575 None admin] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:01:04.035363 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0becf91-cc7b-4466-94ac-f4dcfbb1d575 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:01:04.035636 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0becf91-cc7b-4466-94ac-f4dcfbb1d575 None admin] PATH_INFO: `/v3/projects` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:01:04.036346 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0becf91-cc7b-4466-94ac-f4dcfbb1d575 None admin] RBAC: Authorizing `identity:create_project()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:01:04.037799 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0becf91-cc7b-4466-94ac-f4dcfbb1d575 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:01:04.067431 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 590/1173] 10.222.0.52 () {60 vars in 1141 bytes} [Wed Jul 13 22:01:03 2022] POST /identity/v3/projects => generated 361 bytes in 183 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 22:01:06.121784 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b37068-5032-4c29-926e-acfe2ee86f27 None None] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:01:06.122361 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b37068-5032-4c29-926e-acfe2ee86f27 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:01:06.122361 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46b37068-5032-4c29-926e-acfe2ee86f27 None None] PATH_INFO: `/` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:01:06.123909 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 584/1174] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jul 13 22:01:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 22:01:06.130438 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36191e0f-85e9-48eb-a0a0-6edd1012c148 None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:01:06.131209 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36191e0f-85e9-48eb-a0a0-6edd1012c148 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:01:06.131359 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36191e0f-85e9-48eb-a0a0-6edd1012c148 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:01:06.151621 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-36191e0f-85e9-48eb-a0a0-6edd1012c148 None None] MFA Rules not processed for user `cc99c93da5074cf5a48bc2aa9a92c5e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 22:01:06.152818 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-36191e0f-85e9-48eb-a0a0-6edd1012c148 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:01:06.155720 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:01:06.235116 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 591/1175] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jul 13 22:01:06 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 13 22:01:06.269184 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b37068-5032-4c29-926e-acfe2ee86f27 None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:01:06.388268 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46b37068-5032-4c29-926e-acfe2ee86f27 None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:01:06.427661 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d37b9ef-30e6-4d28-9853-57b6948c7dbf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:01:06.428613 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d37b9ef-30e6-4d28-9853-57b6948c7dbf None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:01:06.428961 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d37b9ef-30e6-4d28-9853-57b6948c7dbf None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:01:06.429145 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d37b9ef-30e6-4d28-9853-57b6948c7dbf None admin] PATH_INFO: `/v3/domains/2bb7deb7d8c140bb85a5e72f18ec3a10` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:01:06.430448 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d37b9ef-30e6-4d28-9853-57b6948c7dbf None admin] RBAC: Authorizing `identity:get_domain(domain_id=2bb7deb7d8c140bb85a5e72f18ec3a10)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:01:06.431936 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d37b9ef-30e6-4d28-9853-57b6948c7dbf None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:01:06.433585 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 585/1176] 10.222.0.52 () {56 vars in 1183 bytes} [Wed Jul 13 22:01:06 2022] GET /identity/v3/domains/2bb7deb7d8c140bb85a5e72f18ec3a10 => generated 267 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 22:01:06.442555 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36191e0f-85e9-48eb-a0a0-6edd1012c148 None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:01:06.552830 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36191e0f-85e9-48eb-a0a0-6edd1012c148 None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:01:06.587503 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0e10469-24af-4c98-87f4-07069c37269e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:01:06.588178 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e10469-24af-4c98-87f4-07069c37269e None admin] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:01:06.588461 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e10469-24af-4c98-87f4-07069c37269e None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:01:06.588740 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0e10469-24af-4c98-87f4-07069c37269e None admin] PATH_INFO: `/v3/users` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:01:06.589475 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e10469-24af-4c98-87f4-07069c37269e None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:01:06.590967 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c0e10469-24af-4c98-87f4-07069c37269e None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:01:06.600773 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:01:06.616184 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 592/1177] 10.222.0.52 () {60 vars in 1132 bytes} [Wed Jul 13 22:01:06 2022] POST /identity/v3/users => generated 311 bytes in 175 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 22:01:08.736550 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2460ea38-ef66-4cb1-9189-e8b68b29973c None None] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:01:08.737175 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2460ea38-ef66-4cb1-9189-e8b68b29973c None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:01:08.737175 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2460ea38-ef66-4cb1-9189-e8b68b29973c None None] PATH_INFO: `/` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:01:08.738483 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 586/1178] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jul 13 22:01:08 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 22:01:08.744253 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e83095-3a3c-44cd-9436-f3bad32dc8ff None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:01:08.744700 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e83095-3a3c-44cd-9436-f3bad32dc8ff None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:01:08.744806 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01e83095-3a3c-44cd-9436-f3bad32dc8ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:01:08.764061 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-01e83095-3a3c-44cd-9436-f3bad32dc8ff None None] MFA Rules not processed for user `cc99c93da5074cf5a48bc2aa9a92c5e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 22:01:08.765238 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-01e83095-3a3c-44cd-9436-f3bad32dc8ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:01:08.843276 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 593/1179] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jul 13 22:01:08 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 13 22:01:08.874871 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2460ea38-ef66-4cb1-9189-e8b68b29973c None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:01:08.921713 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:01:08.997158 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2460ea38-ef66-4cb1-9189-e8b68b29973c None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:01:09.036941 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba39ac1-d79a-41d1-8697-ef2807b00825 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:01:09.037737 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba39ac1-d79a-41d1-8697-ef2807b00825 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:01:09.038064 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba39ac1-d79a-41d1-8697-ef2807b00825 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:01:09.038336 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eba39ac1-d79a-41d1-8697-ef2807b00825 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:01:09.049110 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eba39ac1-d79a-41d1-8697-ef2807b00825 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 22:01:09.056726 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application [None req-eba39ac1-d79a-41d1-8697-ef2807b00825 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 13 22:01:09.056726 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 22:01:09.056726 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 22:01:09.056726 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 22:01:09.056726 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 22:01:09.056726 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 22:01:09.056726 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 22:01:09.056726 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 22:01:09.056726 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 22:01:09.056726 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 22:01:09.056726 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 22:01:09.056726 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 22:01:09.056726 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 22:01:09.056726 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 22:01:09.056726 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 22:01:09.056726 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 22:01:09.057872 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 22:01:09.057872 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 22:01:09.057872 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 22:01:09.057872 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 22:01:09.057872 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 22:01:09.057872 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 22:01:09.057872 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 22:01:09.057872 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 22:01:09.057872 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 22:01:09.057872 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 22:01:09.057872 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 22:01:09.057872 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application with Lock( Jul 13 22:01:09.057872 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 22:01:09.057872 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._enter() Jul 13 22:01:09.057872 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 22:01:09.057872 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 22:01:09.059162 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 22:01:09.059162 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.creator() Jul 13 22:01:09.059162 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 22:01:09.059162 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application created_value = creator( Jul 13 22:01:09.059162 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 22:01:09.059162 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 22:01:09.059162 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 22:01:09.059162 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 22:01:09.059162 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 22:01:09.059162 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 22:01:09.059162 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 13 22:01:09.059162 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Jul 13 22:01:09.059162 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 587/1180] 10.222.0.52 () {56 vars in 1096 bytes} [Wed Jul 13 22:01:08 2022] GET /identity/v3/roles/admin => generated 83 bytes in 184 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 22:01:09.064848 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e83095-3a3c-44cd-9436-f3bad32dc8ff None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:01:09.178928 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01e83095-3a3c-44cd-9436-f3bad32dc8ff None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:01:09.205970 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:01:09.215577 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01530194-9ab1-4733-a8d7-847f076b1a6a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:01:09.216245 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01530194-9ab1-4733-a8d7-847f076b1a6a None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:01:09.216523 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01530194-9ab1-4733-a8d7-847f076b1a6a None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:01:09.216799 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01530194-9ab1-4733-a8d7-847f076b1a6a None admin] PATH_INFO: `/v3/roles` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:01:09.217551 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01530194-9ab1-4733-a8d7-847f076b1a6a None admin] RBAC: Adding query filter params (name=admin) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 22:01:09.217853 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01530194-9ab1-4733-a8d7-847f076b1a6a None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:01:09.219290 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01530194-9ab1-4733-a8d7-847f076b1a6a None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:01:09.228503 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 594/1181] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Jul 13 22:01:09 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 165 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 22:01:09.235201 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba39ac1-d79a-41d1-8697-ef2807b00825 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:01:09.341131 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:01:09.350964 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eba39ac1-d79a-41d1-8697-ef2807b00825 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:01:09.392534 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67916e77-6f14-42ef-b6c6-aff4c18a3b95 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:01:09.393410 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67916e77-6f14-42ef-b6c6-aff4c18a3b95 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:01:09.393690 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67916e77-6f14-42ef-b6c6-aff4c18a3b95 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:01:09.393967 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67916e77-6f14-42ef-b6c6-aff4c18a3b95 None admin] PATH_INFO: `/v3/users/f9d2427c3dca43198e70bec7a7d5ef14` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:01:09.406909 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67916e77-6f14-42ef-b6c6-aff4c18a3b95 None admin] RBAC: Authorizing `identity:get_user(user_id=f9d2427c3dca43198e70bec7a7d5ef14)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:01:09.408367 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67916e77-6f14-42ef-b6c6-aff4c18a3b95 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:01:09.410038 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 588/1182] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Jul 13 22:01:09 2022] GET /identity/v3/users/f9d2427c3dca43198e70bec7a7d5ef14 => generated 311 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 22:01:09.417069 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01530194-9ab1-4733-a8d7-847f076b1a6a None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:01:09.528766 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01530194-9ab1-4733-a8d7-847f076b1a6a None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:01:09.566673 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81f9dbf-07f4-47e9-8792-2b10d7bf516f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:01:09.567340 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81f9dbf-07f4-47e9-8792-2b10d7bf516f None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:01:09.567622 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81f9dbf-07f4-47e9-8792-2b10d7bf516f None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:01:09.567893 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a81f9dbf-07f4-47e9-8792-2b10d7bf516f None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:01:09.628402 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:01:09.641921 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a81f9dbf-07f4-47e9-8792-2b10d7bf516f None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:01:09.643374 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a81f9dbf-07f4-47e9-8792-2b10d7bf516f None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:01:09.758764 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 595/1183] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Jul 13 22:01:09 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 13 22:01:09.765251 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67916e77-6f14-42ef-b6c6-aff4c18a3b95 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:01:09.885489 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67916e77-6f14-42ef-b6c6-aff4c18a3b95 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:01:09.920176 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-795f6d1e-a8fd-413a-af7d-ba72ac62e28b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:01:09.921020 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795f6d1e-a8fd-413a-af7d-ba72ac62e28b None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:01:09.921293 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795f6d1e-a8fd-413a-af7d-ba72ac62e28b None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:01:09.921554 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-795f6d1e-a8fd-413a-af7d-ba72ac62e28b None admin] PATH_INFO: `/v3/projects/88463413d3da41ffaaaa37507c24d65f` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:01:09.922803 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795f6d1e-a8fd-413a-af7d-ba72ac62e28b None admin] RBAC: Authorizing `identity:get_project(project_id=88463413d3da41ffaaaa37507c24d65f)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:01:09.924237 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-795f6d1e-a8fd-413a-af7d-ba72ac62e28b None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:01:09.925917 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 589/1184] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Jul 13 22:01:09 2022] GET /identity/v3/projects/88463413d3da41ffaaaa37507c24d65f => generated 361 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 22:01:09.935584 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81f9dbf-07f4-47e9-8792-2b10d7bf516f None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:01:10.030768 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:01:10.044324 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a81f9dbf-07f4-47e9-8792-2b10d7bf516f None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:01:10.079614 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd570f59-368b-45fd-bec8-868212772ac1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:01:10.080256 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd570f59-368b-45fd-bec8-868212772ac1 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:01:10.080532 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd570f59-368b-45fd-bec8-868212772ac1 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:01:10.080809 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd570f59-368b-45fd-bec8-868212772ac1 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:01:10.081634 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd570f59-368b-45fd-bec8-868212772ac1 None admin] RBAC: Adding query filter params (role.id=44efba2579f342df8a0789c59d163ec3, scope.project.id=88463413d3da41ffaaaa37507c24d65f, user.id=f9d2427c3dca43198e70bec7a7d5ef14) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 22:01:10.081945 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd570f59-368b-45fd-bec8-868212772ac1 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:01:10.083400 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd570f59-368b-45fd-bec8-868212772ac1 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:01:10.089034 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 596/1185] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Jul 13 22:01:09 2022] GET /identity/v3/role_assignments?user.id=f9d2427c3dca43198e70bec7a7d5ef14&scope.project.id=88463413d3da41ffaaaa37507c24d65f&role.id=44efba2579f342df8a0789c59d163ec3 => generated 259 bytes in 154 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 22:01:12.047688 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fca4cf3-20c8-474d-b61b-54e34dc7239d None None] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:01:12.048282 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fca4cf3-20c8-474d-b61b-54e34dc7239d None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:01:12.048282 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fca4cf3-20c8-474d-b61b-54e34dc7239d None None] PATH_INFO: `/` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:01:12.049623 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 590/1186] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jul 13 22:01:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 22:01:12.056030 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca5284b-9e1c-403a-83d9-1198ebf5e021 None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:01:12.056747 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca5284b-9e1c-403a-83d9-1198ebf5e021 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:01:12.056900 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca5284b-9e1c-403a-83d9-1198ebf5e021 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:01:12.079879 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-6ca5284b-9e1c-403a-83d9-1198ebf5e021 None None] MFA Rules not processed for user `cc99c93da5074cf5a48bc2aa9a92c5e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 22:01:12.081074 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-6ca5284b-9e1c-403a-83d9-1198ebf5e021 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:01:12.152392 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 597/1187] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jul 13 22:01:12 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 13 22:01:12.178563 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fca4cf3-20c8-474d-b61b-54e34dc7239d None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:01:12.203854 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:01:12.298314 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fca4cf3-20c8-474d-b61b-54e34dc7239d None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:01:12.337932 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd17e55-cb40-4da3-98c1-2c0c4a16667e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:01:12.338760 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd17e55-cb40-4da3-98c1-2c0c4a16667e None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:01:12.339037 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd17e55-cb40-4da3-98c1-2c0c4a16667e None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:01:12.339310 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbd17e55-cb40-4da3-98c1-2c0c4a16667e None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:01:12.350102 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbd17e55-cb40-4da3-98c1-2c0c4a16667e None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 22:01:12.357752 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application [None req-fbd17e55-cb40-4da3-98c1-2c0c4a16667e None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 13 22:01:12.357752 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 22:01:12.357752 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 22:01:12.357752 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 22:01:12.357752 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 22:01:12.357752 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 22:01:12.357752 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 22:01:12.357752 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 22:01:12.357752 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 22:01:12.357752 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 22:01:12.357752 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 22:01:12.357752 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 22:01:12.357752 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 22:01:12.357752 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 22:01:12.357752 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 22:01:12.357752 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 22:01:12.358891 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 22:01:12.358891 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 22:01:12.358891 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 22:01:12.358891 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 22:01:12.358891 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 22:01:12.358891 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 22:01:12.358891 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 22:01:12.358891 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 22:01:12.358891 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 22:01:12.358891 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 22:01:12.358891 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 22:01:12.358891 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application with Lock( Jul 13 22:01:12.358891 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 22:01:12.358891 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._enter() Jul 13 22:01:12.358891 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 22:01:12.358891 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 22:01:12.359853 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 22:01:12.359853 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.creator() Jul 13 22:01:12.359853 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 22:01:12.359853 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application created_value = creator( Jul 13 22:01:12.359853 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 22:01:12.359853 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 22:01:12.359853 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 22:01:12.359853 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 22:01:12.359853 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 22:01:12.359853 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 22:01:12.359853 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 13 22:01:12.359853 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Jul 13 22:01:12.359853 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 591/1188] 10.222.0.52 () {56 vars in 1096 bytes} [Wed Jul 13 22:01:12 2022] GET /identity/v3/roles/admin => generated 83 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 22:01:12.366000 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca5284b-9e1c-403a-83d9-1198ebf5e021 None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:01:12.454261 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:01:12.473828 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca5284b-9e1c-403a-83d9-1198ebf5e021 None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:01:12.509737 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a06261-48c3-45ce-896c-72d8477702c8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:01:12.510402 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a06261-48c3-45ce-896c-72d8477702c8 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:01:12.510720 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a06261-48c3-45ce-896c-72d8477702c8 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:01:12.510996 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9a06261-48c3-45ce-896c-72d8477702c8 None admin] PATH_INFO: `/v3/roles` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:01:12.511741 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a06261-48c3-45ce-896c-72d8477702c8 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 22:01:12.512105 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a06261-48c3-45ce-896c-72d8477702c8 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:01:12.513496 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e9a06261-48c3-45ce-896c-72d8477702c8 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:01:12.521899 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 598/1189] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Jul 13 22:01:12 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 22:01:12.531182 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd17e55-cb40-4da3-98c1-2c0c4a16667e None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:01:12.642609 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:01:12.664894 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbd17e55-cb40-4da3-98c1-2c0c4a16667e None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:01:12.699033 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a81115f-1d4d-4eb8-900b-b018b0eb4e1c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:01:12.699934 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a81115f-1d4d-4eb8-900b-b018b0eb4e1c None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:01:12.700213 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a81115f-1d4d-4eb8-900b-b018b0eb4e1c None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:01:12.700482 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a81115f-1d4d-4eb8-900b-b018b0eb4e1c None admin] PATH_INFO: `/v3/users/f9d2427c3dca43198e70bec7a7d5ef14` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:01:12.701777 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a81115f-1d4d-4eb8-900b-b018b0eb4e1c None admin] RBAC: Authorizing `identity:get_user(user_id=f9d2427c3dca43198e70bec7a7d5ef14)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:01:12.703247 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3a81115f-1d4d-4eb8-900b-b018b0eb4e1c None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:01:12.704892 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 592/1190] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Jul 13 22:01:12 2022] GET /identity/v3/users/f9d2427c3dca43198e70bec7a7d5ef14 => generated 311 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 22:01:12.711939 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a06261-48c3-45ce-896c-72d8477702c8 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:01:12.819490 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9a06261-48c3-45ce-896c-72d8477702c8 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:01:12.853948 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02a1ec53-c58a-45b7-a147-f4618d97f1bd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:01:12.854835 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a1ec53-c58a-45b7-a147-f4618d97f1bd None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:01:12.855114 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a1ec53-c58a-45b7-a147-f4618d97f1bd None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:01:12.855387 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02a1ec53-c58a-45b7-a147-f4618d97f1bd None admin] PATH_INFO: `/v3/projects/88463413d3da41ffaaaa37507c24d65f` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:01:12.856577 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a1ec53-c58a-45b7-a147-f4618d97f1bd None admin] RBAC: Authorizing `identity:get_project(project_id=88463413d3da41ffaaaa37507c24d65f)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:01:12.858013 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02a1ec53-c58a-45b7-a147-f4618d97f1bd None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:01:12.859738 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 599/1191] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Jul 13 22:01:12 2022] GET /identity/v3/projects/88463413d3da41ffaaaa37507c24d65f => generated 361 bytes in 149 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 22:01:12.865816 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a81115f-1d4d-4eb8-900b-b018b0eb4e1c None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:01:12.978158 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a81115f-1d4d-4eb8-900b-b018b0eb4e1c None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:01:13.012766 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a7ea37-76f8-4782-9301-ce4504652bd7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:01:13.013648 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a7ea37-76f8-4782-9301-ce4504652bd7 None admin] REQUEST_METHOD: `PUT` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:01:13.013921 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a7ea37-76f8-4782-9301-ce4504652bd7 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:01:13.014187 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a7ea37-76f8-4782-9301-ce4504652bd7 None admin] PATH_INFO: `/v3/projects/88463413d3da41ffaaaa37507c24d65f/users/f9d2427c3dca43198e70bec7a7d5ef14/roles/44efba2579f342df8a0789c59d163ec3` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:01:13.016264 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a7ea37-76f8-4782-9301-ce4504652bd7 None admin] RBAC: Authorizing `identity:create_grant(project_id=88463413d3da41ffaaaa37507c24d65f, user_id=f9d2427c3dca43198e70bec7a7d5ef14, role_id=44efba2579f342df8a0789c59d163ec3)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:01:13.017699 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a7ea37-76f8-4782-9301-ce4504652bd7 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:01:13.033936 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 593/1192] 10.222.0.52 () {58 vars in 1439 bytes} [Wed Jul 13 22:01:12 2022] PUT /identity/v3/projects/88463413d3da41ffaaaa37507c24d65f/users/f9d2427c3dca43198e70bec7a7d5ef14/roles/44efba2579f342df8a0789c59d163ec3 => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 22:01:15.005050 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be869e0e-c6cb-4af0-bd45-04e350bf834f None None] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:01:15.005905 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be869e0e-c6cb-4af0-bd45-04e350bf834f None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:01:15.005905 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be869e0e-c6cb-4af0-bd45-04e350bf834f None None] PATH_INFO: `/` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:01:15.008044 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 600/1193] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jul 13 22:01:15 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 22:01:15.014288 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4484b3b4-47d8-4a0b-adb2-47e4f320bd31 None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:01:15.014779 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4484b3b4-47d8-4a0b-adb2-47e4f320bd31 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:01:15.014881 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4484b3b4-47d8-4a0b-adb2-47e4f320bd31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:01:15.041055 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-4484b3b4-47d8-4a0b-adb2-47e4f320bd31 None None] MFA Rules not processed for user `cc99c93da5074cf5a48bc2aa9a92c5e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 22:01:15.042231 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-4484b3b4-47d8-4a0b-adb2-47e4f320bd31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:01:15.044285 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:01:15.113065 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 594/1194] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jul 13 22:01:15 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 13 22:01:15.144151 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be869e0e-c6cb-4af0-bd45-04e350bf834f None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:01:15.160480 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:01:15.260872 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be869e0e-c6cb-4af0-bd45-04e350bf834f None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:01:15.295820 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e6495d-bac1-4183-acc9-e7a75e92fbbd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:01:15.296635 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e6495d-bac1-4183-acc9-e7a75e92fbbd None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:01:15.296915 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e6495d-bac1-4183-acc9-e7a75e92fbbd None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:01:15.297274 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e6495d-bac1-4183-acc9-e7a75e92fbbd None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:01:15.307830 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e6495d-bac1-4183-acc9-e7a75e92fbbd None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 22:01:15.315454 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application [None req-d7e6495d-bac1-4183-acc9-e7a75e92fbbd None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 13 22:01:15.315454 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 22:01:15.315454 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 22:01:15.315454 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 22:01:15.315454 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 22:01:15.315454 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 22:01:15.315454 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 22:01:15.315454 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 22:01:15.315454 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 22:01:15.315454 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 22:01:15.315454 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 22:01:15.315454 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 22:01:15.315454 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 22:01:15.315454 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 22:01:15.315454 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 22:01:15.315454 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 22:01:15.316491 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 22:01:15.316491 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 22:01:15.316491 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 22:01:15.316491 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 22:01:15.316491 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 22:01:15.316491 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 22:01:15.316491 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 22:01:15.316491 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 22:01:15.316491 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 22:01:15.316491 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 22:01:15.316491 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 22:01:15.316491 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application with Lock( Jul 13 22:01:15.316491 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 22:01:15.316491 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._enter() Jul 13 22:01:15.316491 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 22:01:15.316491 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 22:01:15.317353 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 22:01:15.317353 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.creator() Jul 13 22:01:15.317353 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 22:01:15.317353 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application created_value = creator( Jul 13 22:01:15.317353 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 22:01:15.317353 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 22:01:15.317353 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 22:01:15.317353 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 22:01:15.317353 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 22:01:15.317353 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 22:01:15.317353 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 13 22:01:15.317353 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Jul 13 22:01:15.317353 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 601/1195] 10.222.0.52 () {56 vars in 1096 bytes} [Wed Jul 13 22:01:15 2022] GET /identity/v3/roles/admin => generated 83 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 22:01:15.322989 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4484b3b4-47d8-4a0b-adb2-47e4f320bd31 None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:01:15.426243 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4484b3b4-47d8-4a0b-adb2-47e4f320bd31 None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:01:15.462700 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3754da6-1962-4fb9-8e24-790c2c653d62 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:01:15.463359 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3754da6-1962-4fb9-8e24-790c2c653d62 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:01:15.463651 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3754da6-1962-4fb9-8e24-790c2c653d62 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:01:15.463901 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3754da6-1962-4fb9-8e24-790c2c653d62 None admin] PATH_INFO: `/v3/roles` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:01:15.464651 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3754da6-1962-4fb9-8e24-790c2c653d62 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 22:01:15.464951 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3754da6-1962-4fb9-8e24-790c2c653d62 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:01:15.466323 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3754da6-1962-4fb9-8e24-790c2c653d62 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:01:15.471308 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:01:15.474862 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 595/1196] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Jul 13 22:01:15 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 22:01:15.481615 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e6495d-bac1-4183-acc9-e7a75e92fbbd None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:01:15.561567 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:01:15.592967 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e6495d-bac1-4183-acc9-e7a75e92fbbd None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:01:15.630967 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8bcdd9-c5a9-4120-befa-cd2880edd104 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:01:15.631846 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8bcdd9-c5a9-4120-befa-cd2880edd104 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:01:15.632124 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8bcdd9-c5a9-4120-befa-cd2880edd104 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:01:15.632541 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed8bcdd9-c5a9-4120-befa-cd2880edd104 None admin] PATH_INFO: `/v3/users/f9d2427c3dca43198e70bec7a7d5ef14` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:01:15.633861 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8bcdd9-c5a9-4120-befa-cd2880edd104 None admin] RBAC: Authorizing `identity:get_user(user_id=f9d2427c3dca43198e70bec7a7d5ef14)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:01:15.635345 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed8bcdd9-c5a9-4120-befa-cd2880edd104 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:01:15.636996 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 602/1197] 10.222.0.52 () {56 vars in 1177 bytes} [Wed Jul 13 22:01:15 2022] GET /identity/v3/users/f9d2427c3dca43198e70bec7a7d5ef14 => generated 311 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 22:01:15.643910 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3754da6-1962-4fb9-8e24-790c2c653d62 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:01:15.756654 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3754da6-1962-4fb9-8e24-790c2c653d62 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:01:15.802101 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603c5b51-9257-47fc-ab84-73533afff428 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:01:15.802711 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603c5b51-9257-47fc-ab84-73533afff428 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:01:15.802989 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603c5b51-9257-47fc-ab84-73533afff428 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:01:15.803267 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603c5b51-9257-47fc-ab84-73533afff428 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:01:15.871673 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:01:15.874191 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603c5b51-9257-47fc-ab84-73533afff428 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:01:15.875779 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-603c5b51-9257-47fc-ab84-73533afff428 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:01:15.978186 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 596/1198] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Jul 13 22:01:15 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 336 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 13 22:01:15.987735 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8bcdd9-c5a9-4120-befa-cd2880edd104 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:01:16.099060 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed8bcdd9-c5a9-4120-befa-cd2880edd104 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:01:16.133884 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d632a3ba-fe11-4773-afdd-74565274d5fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:01:16.134762 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d632a3ba-fe11-4773-afdd-74565274d5fd None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:01:16.135036 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d632a3ba-fe11-4773-afdd-74565274d5fd None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:01:16.135306 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d632a3ba-fe11-4773-afdd-74565274d5fd None admin] PATH_INFO: `/v3/projects/88463413d3da41ffaaaa37507c24d65f` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:01:16.136570 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d632a3ba-fe11-4773-afdd-74565274d5fd None admin] RBAC: Authorizing `identity:get_project(project_id=88463413d3da41ffaaaa37507c24d65f)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:01:16.138012 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d632a3ba-fe11-4773-afdd-74565274d5fd None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:01:16.139754 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 603/1199] 10.222.0.52 () {56 vars in 1186 bytes} [Wed Jul 13 22:01:15 2022] GET /identity/v3/projects/88463413d3da41ffaaaa37507c24d65f => generated 361 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 22:01:16.146191 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603c5b51-9257-47fc-ab84-73533afff428 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:01:16.248607 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:01:16.251694 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603c5b51-9257-47fc-ab84-73533afff428 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:01:16.289252 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f7e69cb-f1df-4075-a686-268b9aded10f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:01:16.290306 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7e69cb-f1df-4075-a686-268b9aded10f None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:01:16.290636 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7e69cb-f1df-4075-a686-268b9aded10f None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:01:16.290900 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f7e69cb-f1df-4075-a686-268b9aded10f None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:01:16.291741 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7e69cb-f1df-4075-a686-268b9aded10f None admin] RBAC: Adding query filter params (role.id=44efba2579f342df8a0789c59d163ec3, scope.project.id=88463413d3da41ffaaaa37507c24d65f, user.id=f9d2427c3dca43198e70bec7a7d5ef14) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 22:01:16.292053 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7e69cb-f1df-4075-a686-268b9aded10f None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:01:16.293482 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f7e69cb-f1df-4075-a686-268b9aded10f None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:01:16.302708 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 597/1200] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Jul 13 22:01:16 2022] GET /identity/v3/role_assignments?user.id=f9d2427c3dca43198e70bec7a7d5ef14&scope.project.id=88463413d3da41ffaaaa37507c24d65f&role.id=44efba2579f342df8a0789c59d163ec3 => generated 608 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 22:01:26.128447 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=96 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:01:26.297634 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=18 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:02:11.885232 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bcc029-dbbf-4055-a350-6755bb525ade None None] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:02:11.885987 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bcc029-dbbf-4055-a350-6755bb525ade None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:02:11.885987 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61bcc029-dbbf-4055-a350-6755bb525ade None None] PATH_INFO: `/` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:02:11.887325 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 604/1201] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jul 13 22:02:11 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 22:02:11.892792 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae9108b-e6a5-4b13-a4fc-91a23e583cd0 None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:02:11.893249 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae9108b-e6a5-4b13-a4fc-91a23e583cd0 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:02:11.893361 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cae9108b-e6a5-4b13-a4fc-91a23e583cd0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:02:11.914222 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-cae9108b-e6a5-4b13-a4fc-91a23e583cd0 None None] MFA Rules not processed for user `cc99c93da5074cf5a48bc2aa9a92c5e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 22:02:11.915602 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-cae9108b-e6a5-4b13-a4fc-91a23e583cd0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:02:12.005998 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 598/1202] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jul 13 22:02:11 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 13 22:02:12.024043 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61bcc029-dbbf-4055-a350-6755bb525ade None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:02:12.163829 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61bcc029-dbbf-4055-a350-6755bb525ade None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:02:12.209157 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee4c40b-f11f-471c-b003-dca6e3931005 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:02:12.210081 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee4c40b-f11f-471c-b003-dca6e3931005 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:02:12.210443 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee4c40b-f11f-471c-b003-dca6e3931005 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:02:12.210733 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ee4c40b-f11f-471c-b003-dca6e3931005 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:02:12.224793 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee4c40b-f11f-471c-b003-dca6e3931005 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:02:12.226449 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ee4c40b-f11f-471c-b003-dca6e3931005 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:02:12.249650 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application [None req-5ee4c40b-f11f-471c-b003-dca6e3931005 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 22:02:12.249650 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 22:02:12.249650 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 22:02:12.249650 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 22:02:12.249650 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 22:02:12.249650 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 22:02:12.249650 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 22:02:12.249650 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 22:02:12.249650 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 22:02:12.249650 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 22:02:12.249650 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 22:02:12.249650 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 22:02:12.249650 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 22:02:12.249650 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 22:02:12.249650 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 22:02:12.256395 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 22:02:12.256395 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 22:02:12.256395 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 22:02:12.256395 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 22:02:12.256395 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 22:02:12.256395 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 22:02:12.256395 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 22:02:12.256395 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 22:02:12.256395 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application with Lock( Jul 13 22:02:12.256395 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 22:02:12.256395 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._enter() Jul 13 22:02:12.256395 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 22:02:12.256395 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 22:02:12.256395 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 22:02:12.256395 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.creator() Jul 13 22:02:12.256395 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 22:02:12.256395 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application created_value = creator( Jul 13 22:02:12.257444 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 22:02:12.257444 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 22:02:12.257444 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 22:02:12.257444 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Jul 13 22:02:12.257444 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 605/1203] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Jul 13 22:02:12 2022] GET /identity/v3/domains/Default => generated 87 bytes in 228 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 22:02:12.257803 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae9108b-e6a5-4b13-a4fc-91a23e583cd0 None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:02:12.395074 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cae9108b-e6a5-4b13-a4fc-91a23e583cd0 None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:02:12.431677 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:02:12.439084 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d56bb69c-1387-4b3e-beba-e9c2a00ad121 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:02:12.439761 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56bb69c-1387-4b3e-beba-e9c2a00ad121 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:02:12.440039 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56bb69c-1387-4b3e-beba-e9c2a00ad121 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:02:12.440310 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d56bb69c-1387-4b3e-beba-e9c2a00ad121 None admin] PATH_INFO: `/v3/domains` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:02:12.441056 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56bb69c-1387-4b3e-beba-e9c2a00ad121 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 22:02:12.441354 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56bb69c-1387-4b3e-beba-e9c2a00ad121 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:02:12.442953 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d56bb69c-1387-4b3e-beba-e9c2a00ad121 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:02:12.463290 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 599/1204] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Jul 13 22:02:12 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 202 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 22:02:12.464840 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee4c40b-f11f-471c-b003-dca6e3931005 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:02:12.542835 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:02:12.608320 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ee4c40b-f11f-471c-b003-dca6e3931005 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:02:12.650879 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f645b5f-d195-4856-8401-0f08a328ed17 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:02:12.651605 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f645b5f-d195-4856-8401-0f08a328ed17 None admin] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:02:12.651875 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f645b5f-d195-4856-8401-0f08a328ed17 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:02:12.652144 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f645b5f-d195-4856-8401-0f08a328ed17 None admin] PATH_INFO: `/v3/users` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:02:12.652892 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f645b5f-d195-4856-8401-0f08a328ed17 None admin] RBAC: Authorizing `identity:create_user()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:02:12.654351 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f645b5f-d195-4856-8401-0f08a328ed17 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:02:12.686598 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 606/1205] 10.222.0.52 () {60 vars in 1132 bytes} [Wed Jul 13 22:02:12 2022] POST /identity/v3/users => generated 251 bytes in 215 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 22:02:15.000054 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b99484-18d6-4223-b7af-4d23c162bee4 None None] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:02:15.000650 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b99484-18d6-4223-b7af-4d23c162bee4 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:02:15.000650 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75b99484-18d6-4223-b7af-4d23c162bee4 None None] PATH_INFO: `/` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:02:15.003011 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 600/1206] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jul 13 22:02:14 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 22:02:15.009053 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6d1058-9341-437e-947d-6c6f0d6ea805 None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:02:15.009525 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6d1058-9341-437e-947d-6c6f0d6ea805 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:02:15.009635 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f6d1058-9341-437e-947d-6c6f0d6ea805 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:02:15.028673 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-6f6d1058-9341-437e-947d-6c6f0d6ea805 None None] MFA Rules not processed for user `cc99c93da5074cf5a48bc2aa9a92c5e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 22:02:15.030014 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-6f6d1058-9341-437e-947d-6c6f0d6ea805 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:02:15.110855 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 607/1207] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jul 13 22:02:15 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 13 22:02:15.135459 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b99484-18d6-4223-b7af-4d23c162bee4 None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:02:15.272156 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75b99484-18d6-4223-b7af-4d23c162bee4 None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:02:15.312943 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396dcb1a-2085-460b-896f-26b656ef4ca0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:02:15.314093 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396dcb1a-2085-460b-896f-26b656ef4ca0 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:02:15.314438 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396dcb1a-2085-460b-896f-26b656ef4ca0 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:02:15.314765 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-396dcb1a-2085-460b-896f-26b656ef4ca0 None admin] PATH_INFO: `/v3/roles/service` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:02:15.328992 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-396dcb1a-2085-460b-896f-26b656ef4ca0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 22:02:15.338166 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application [None req-396dcb1a-2085-460b-896f-26b656ef4ca0 None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 13 22:02:15.338166 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 22:02:15.338166 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 22:02:15.338166 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 22:02:15.338166 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 22:02:15.338166 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 22:02:15.338166 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 22:02:15.338166 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 22:02:15.338166 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 22:02:15.338166 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 22:02:15.338166 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 22:02:15.338166 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 22:02:15.338166 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 22:02:15.338166 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 22:02:15.338166 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 22:02:15.338166 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 22:02:15.339228 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 22:02:15.339228 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 22:02:15.339228 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 22:02:15.339228 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 22:02:15.339228 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 22:02:15.339228 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 22:02:15.339228 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 22:02:15.339228 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 22:02:15.339228 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 22:02:15.339228 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 22:02:15.339228 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 22:02:15.339228 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application with Lock( Jul 13 22:02:15.339228 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 22:02:15.339228 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._enter() Jul 13 22:02:15.339228 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 22:02:15.339228 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 22:02:15.340148 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 22:02:15.340148 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.creator() Jul 13 22:02:15.340148 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 22:02:15.340148 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application created_value = creator( Jul 13 22:02:15.340148 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 22:02:15.340148 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 22:02:15.340148 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 22:02:15.340148 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 22:02:15.340148 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 22:02:15.340148 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 22:02:15.340148 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 13 22:02:15.340148 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Jul 13 22:02:15.340148 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 601/1208] 10.222.0.52 () {56 vars in 1102 bytes} [Wed Jul 13 22:02:15 2022] GET /identity/v3/roles/service => generated 85 bytes in 205 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 22:02:15.345727 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6d1058-9341-437e-947d-6c6f0d6ea805 None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:02:15.389516 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:02:15.457911 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f6d1058-9341-437e-947d-6c6f0d6ea805 None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:02:15.503344 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3a5679-4762-4829-b164-d5f7a05415aa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:02:15.504111 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3a5679-4762-4829-b164-d5f7a05415aa None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:02:15.504550 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3a5679-4762-4829-b164-d5f7a05415aa None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:02:15.504828 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d3a5679-4762-4829-b164-d5f7a05415aa None admin] PATH_INFO: `/v3/roles` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:02:15.505650 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3a5679-4762-4829-b164-d5f7a05415aa None admin] RBAC: Adding query filter params (name=service) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 22:02:15.505956 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3a5679-4762-4829-b164-d5f7a05415aa None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:02:15.507684 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d3a5679-4762-4829-b164-d5f7a05415aa None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:02:15.517492 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 608/1209] 10.222.0.52 () {56 vars in 1103 bytes} [Wed Jul 13 22:02:15 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 22:02:15.522970 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396dcb1a-2085-460b-896f-26b656ef4ca0 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:02:15.587878 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:02:15.655570 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-396dcb1a-2085-460b-896f-26b656ef4ca0 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:02:15.693453 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020f5376-14df-4cf3-8664-8e80d4409876 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:02:15.694629 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020f5376-14df-4cf3-8664-8e80d4409876 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:02:15.695196 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020f5376-14df-4cf3-8664-8e80d4409876 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:02:15.695615 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-020f5376-14df-4cf3-8664-8e80d4409876 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:02:15.709478 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-020f5376-14df-4cf3-8664-8e80d4409876 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:02:15.711170 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-020f5376-14df-4cf3-8664-8e80d4409876 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:02:15.731638 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application [None req-020f5376-14df-4cf3-8664-8e80d4409876 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 22:02:15.731638 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 22:02:15.731638 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 22:02:15.731638 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 22:02:15.731638 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 22:02:15.731638 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 22:02:15.731638 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 22:02:15.731638 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 22:02:15.731638 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 22:02:15.731638 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 22:02:15.731638 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 22:02:15.731638 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 22:02:15.731638 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 22:02:15.731638 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 22:02:15.731638 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 22:02:15.732846 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 22:02:15.732846 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 22:02:15.732846 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 22:02:15.732846 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 22:02:15.732846 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 22:02:15.732846 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 22:02:15.732846 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 22:02:15.732846 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 22:02:15.732846 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application with Lock( Jul 13 22:02:15.732846 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 22:02:15.732846 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._enter() Jul 13 22:02:15.732846 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 22:02:15.732846 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 22:02:15.732846 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 22:02:15.732846 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.creator() Jul 13 22:02:15.732846 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 22:02:15.732846 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application created_value = creator( Jul 13 22:02:15.733899 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 22:02:15.733899 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 22:02:15.733899 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 22:02:15.733899 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Jul 13 22:02:15.733899 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 602/1210] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Jul 13 22:02:15 2022] GET /identity/v3/domains/Default => generated 87 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 13 22:02:15.739031 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3a5679-4762-4829-b164-d5f7a05415aa None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:02:15.858038 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:02:15.862200 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d3a5679-4762-4829-b164-d5f7a05415aa None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:02:15.911580 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9805973e-fb5b-4bce-b7c4-a0806b58d0b7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:02:15.912418 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9805973e-fb5b-4bce-b7c4-a0806b58d0b7 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:02:15.912682 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9805973e-fb5b-4bce-b7c4-a0806b58d0b7 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:02:15.912961 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9805973e-fb5b-4bce-b7c4-a0806b58d0b7 None admin] PATH_INFO: `/v3/domains` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:02:15.913696 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9805973e-fb5b-4bce-b7c4-a0806b58d0b7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 22:02:15.914057 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9805973e-fb5b-4bce-b7c4-a0806b58d0b7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:02:15.915598 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9805973e-fb5b-4bce-b7c4-a0806b58d0b7 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:02:15.932144 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 609/1211] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Jul 13 22:02:15 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 194 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 22:02:15.938575 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020f5376-14df-4cf3-8664-8e80d4409876 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:02:16.050443 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-020f5376-14df-4cf3-8664-8e80d4409876 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:02:16.052088 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:02:16.089240 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6e7a5d-d6a6-4b6d-b020-931ab286a1b1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:02:16.090150 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6e7a5d-d6a6-4b6d-b020-931ab286a1b1 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:02:16.090504 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6e7a5d-d6a6-4b6d-b020-931ab286a1b1 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:02:16.090852 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf6e7a5d-d6a6-4b6d-b020-931ab286a1b1 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:02:16.099430 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6e7a5d-d6a6-4b6d-b020-931ab286a1b1 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:02:16.101001 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf6e7a5d-d6a6-4b6d-b020-931ab286a1b1 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:02:16.116579 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application [None req-bf6e7a5d-d6a6-4b6d-b020-931ab286a1b1 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 13 22:02:16.116579 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 22:02:16.116579 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 22:02:16.116579 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 22:02:16.116579 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 22:02:16.116579 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 22:02:16.116579 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 22:02:16.116579 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 22:02:16.116579 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 22:02:16.116579 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 22:02:16.116579 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 22:02:16.116579 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 22:02:16.116579 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 13 22:02:16.116579 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 13 22:02:16.116579 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 13 22:02:16.116579 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 13 22:02:16.117919 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 22:02:16.117919 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 22:02:16.117919 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 13 22:02:16.117919 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 22:02:16.117919 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 13 22:02:16.117919 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 22:02:16.117919 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 22:02:16.117919 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 22:02:16.117919 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 22:02:16.117919 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 22:02:16.117919 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 22:02:16.117919 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application with Lock( Jul 13 22:02:16.117919 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 22:02:16.117919 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._enter() Jul 13 22:02:16.117919 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 22:02:16.117919 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 22:02:16.117919 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 22:02:16.117919 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.creator() Jul 13 22:02:16.123779 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 22:02:16.123779 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application created_value = creator( Jul 13 22:02:16.123779 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 13 22:02:16.123779 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 13 22:02:16.123779 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 13 22:02:16.123779 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 13 22:02:16.123779 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 13 22:02:16.123779 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 13 22:02:16.123779 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 13 22:02:16.123779 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Jul 13 22:02:16.123779 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 603/1212] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Jul 13 22:02:15 2022] GET /identity/v3/users/placement => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 22:02:16.124518 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9805973e-fb5b-4bce-b7c4-a0806b58d0b7 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:02:16.244929 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9805973e-fb5b-4bce-b7c4-a0806b58d0b7 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:02:16.282756 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5216d5-b982-45d9-b5a1-dc415bd29a94 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:02:16.283567 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5216d5-b982-45d9-b5a1-dc415bd29a94 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:02:16.283848 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5216d5-b982-45d9-b5a1-dc415bd29a94 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:02:16.284132 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb5216d5-b982-45d9-b5a1-dc415bd29a94 None admin] PATH_INFO: `/v3/users` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:02:16.285108 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5216d5-b982-45d9-b5a1-dc415bd29a94 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 22:02:16.285420 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5216d5-b982-45d9-b5a1-dc415bd29a94 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:02:16.287078 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb5216d5-b982-45d9-b5a1-dc415bd29a94 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:02:16.311351 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 610/1213] 10.222.0.52 () {56 vars in 1143 bytes} [Wed Jul 13 22:02:16 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 183 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 22:02:16.315767 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6e7a5d-d6a6-4b6d-b020-931ab286a1b1 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:02:16.434138 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf6e7a5d-d6a6-4b6d-b020-931ab286a1b1 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:02:16.474219 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780e934b-e56e-4075-9077-24a6873919c2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:02:16.474917 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780e934b-e56e-4075-9077-24a6873919c2 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:02:16.475236 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780e934b-e56e-4075-9077-24a6873919c2 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:02:16.475548 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-780e934b-e56e-4075-9077-24a6873919c2 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:02:16.512225 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:02:16.566170 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780e934b-e56e-4075-9077-24a6873919c2 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:02:16.567898 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-780e934b-e56e-4075-9077-24a6873919c2 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:02:16.684742 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 604/1214] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Jul 13 22:02:16 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 373 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 13 22:02:16.690452 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5216d5-b982-45d9-b5a1-dc415bd29a94 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:02:16.712948 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:02:16.811392 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb5216d5-b982-45d9-b5a1-dc415bd29a94 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:02:16.852951 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8503f0f6-6d98-463d-9432-8703e6a174db None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:02:16.853901 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8503f0f6-6d98-463d-9432-8703e6a174db None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:02:16.854186 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8503f0f6-6d98-463d-9432-8703e6a174db None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:02:16.854661 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8503f0f6-6d98-463d-9432-8703e6a174db None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:02:16.866743 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8503f0f6-6d98-463d-9432-8703e6a174db None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:02:16.868727 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8503f0f6-6d98-463d-9432-8703e6a174db None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:02:16.889950 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application [None req-8503f0f6-6d98-463d-9432-8703e6a174db None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 22:02:16.889950 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 22:02:16.889950 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 22:02:16.889950 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 22:02:16.889950 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 22:02:16.889950 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 22:02:16.889950 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 22:02:16.889950 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 22:02:16.889950 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 22:02:16.889950 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 22:02:16.889950 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 22:02:16.889950 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 22:02:16.889950 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 22:02:16.889950 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 22:02:16.889950 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 22:02:16.891436 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 22:02:16.891436 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 22:02:16.891436 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 22:02:16.891436 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 22:02:16.891436 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 22:02:16.891436 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 22:02:16.891436 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 22:02:16.891436 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 22:02:16.891436 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application with Lock( Jul 13 22:02:16.891436 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 22:02:16.891436 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._enter() Jul 13 22:02:16.891436 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 22:02:16.891436 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 22:02:16.891436 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 22:02:16.891436 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.creator() Jul 13 22:02:16.891436 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 22:02:16.891436 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application created_value = creator( Jul 13 22:02:16.892528 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 22:02:16.892528 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 22:02:16.892528 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 22:02:16.892528 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Jul 13 22:02:16.896380 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 611/1215] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Jul 13 22:02:16 2022] GET /identity/v3/domains/Default => generated 87 bytes in 206 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 13 22:02:16.897566 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780e934b-e56e-4075-9077-24a6873919c2 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:02:16.982453 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:02:17.028405 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-780e934b-e56e-4075-9077-24a6873919c2 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:02:17.065263 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abcedaf3-c96b-4269-bf5f-a3149b314cdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:02:17.066772 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcedaf3-c96b-4269-bf5f-a3149b314cdd None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:02:17.067202 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcedaf3-c96b-4269-bf5f-a3149b314cdd None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:02:17.067661 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-abcedaf3-c96b-4269-bf5f-a3149b314cdd None admin] PATH_INFO: `/v3/domains` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:02:17.068397 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abcedaf3-c96b-4269-bf5f-a3149b314cdd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 22:02:17.068705 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abcedaf3-c96b-4269-bf5f-a3149b314cdd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:02:17.070174 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-abcedaf3-c96b-4269-bf5f-a3149b314cdd None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:02:17.086708 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 605/1216] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Jul 13 22:02:16 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 186 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 22:02:17.089372 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8503f0f6-6d98-463d-9432-8703e6a174db None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:02:17.169709 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:02:17.205293 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8503f0f6-6d98-463d-9432-8703e6a174db None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:02:17.243470 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98827999-b86b-43a7-8324-5615c03a15d2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:02:17.244332 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98827999-b86b-43a7-8324-5615c03a15d2 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:02:17.244655 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98827999-b86b-43a7-8324-5615c03a15d2 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:02:17.244996 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98827999-b86b-43a7-8324-5615c03a15d2 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:02:17.252275 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98827999-b86b-43a7-8324-5615c03a15d2 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:02:17.253747 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-98827999-b86b-43a7-8324-5615c03a15d2 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:02:17.269328 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application [None req-98827999-b86b-43a7-8324-5615c03a15d2 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 13 22:02:17.269328 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 22:02:17.269328 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 22:02:17.269328 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 22:02:17.269328 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 22:02:17.269328 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 22:02:17.269328 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 22:02:17.269328 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 22:02:17.269328 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 22:02:17.269328 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 22:02:17.269328 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 22:02:17.269328 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 22:02:17.269328 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 13 22:02:17.269328 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 13 22:02:17.269328 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 13 22:02:17.270524 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 13 22:02:17.270524 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 22:02:17.270524 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 22:02:17.270524 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 22:02:17.270524 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 22:02:17.270524 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 22:02:17.270524 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 22:02:17.270524 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 22:02:17.270524 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application with Lock( Jul 13 22:02:17.270524 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 22:02:17.270524 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._enter() Jul 13 22:02:17.270524 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 22:02:17.270524 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 22:02:17.270524 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 22:02:17.270524 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.creator() Jul 13 22:02:17.270524 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 22:02:17.270524 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application created_value = creator( Jul 13 22:02:17.271588 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 13 22:02:17.271588 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 13 22:02:17.271588 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 13 22:02:17.271588 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 13 22:02:17.271588 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 13 22:02:17.271588 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 13 22:02:17.271588 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 13 22:02:17.271588 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Jul 13 22:02:17.271588 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 612/1217] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Jul 13 22:02:17 2022] GET /identity/v3/projects/service => generated 88 bytes in 182 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 22:02:17.277124 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abcedaf3-c96b-4269-bf5f-a3149b314cdd None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:02:17.391088 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-abcedaf3-c96b-4269-bf5f-a3149b314cdd None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:02:17.418268 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:02:17.432334 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbba8c74-ce06-4e1a-972e-c668525df912 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:02:17.433202 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbba8c74-ce06-4e1a-972e-c668525df912 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:02:17.436062 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbba8c74-ce06-4e1a-972e-c668525df912 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:02:17.436062 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbba8c74-ce06-4e1a-972e-c668525df912 None admin] PATH_INFO: `/v3/projects` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:02:17.436062 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbba8c74-ce06-4e1a-972e-c668525df912 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 22:02:17.436062 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbba8c74-ce06-4e1a-972e-c668525df912 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:02:17.436657 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbba8c74-ce06-4e1a-972e-c668525df912 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:02:17.455238 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 606/1218] 10.222.0.52 () {56 vars in 1148 bytes} [Wed Jul 13 22:02:17 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 174 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 22:02:17.457565 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98827999-b86b-43a7-8324-5615c03a15d2 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:02:17.574879 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98827999-b86b-43a7-8324-5615c03a15d2 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:02:17.604209 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:02:17.611803 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-430692e8-fc16-4cc6-b19b-3096481d59b2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:02:17.612497 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430692e8-fc16-4cc6-b19b-3096481d59b2 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:02:17.612800 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430692e8-fc16-4cc6-b19b-3096481d59b2 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:02:17.613077 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-430692e8-fc16-4cc6-b19b-3096481d59b2 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:02:17.614027 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430692e8-fc16-4cc6-b19b-3096481d59b2 None admin] RBAC: Adding query filter params (role.id=70c34ea493484322911dab62b4a56e75, scope.project.id=417e3e2f1333481795a089bef2e24bea, user.id=ac673d641d8849c79bf4cf71c9805587) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 22:02:17.614317 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430692e8-fc16-4cc6-b19b-3096481d59b2 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:02:17.615822 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-430692e8-fc16-4cc6-b19b-3096481d59b2 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:02:17.622920 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 613/1219] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Jul 13 22:02:17 2022] GET /identity/v3/role_assignments?user.id=ac673d641d8849c79bf4cf71c9805587&scope.project.id=417e3e2f1333481795a089bef2e24bea&role.id=70c34ea493484322911dab62b4a56e75 => generated 259 bytes in 166 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 22:02:19.749437 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e95667-1e60-443c-a8d6-20abc95066dd None None] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:02:19.750011 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e95667-1e60-443c-a8d6-20abc95066dd None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:02:19.750011 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36e95667-1e60-443c-a8d6-20abc95066dd None None] PATH_INFO: `/` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:02:19.751415 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 607/1220] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jul 13 22:02:19 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 22:02:19.756664 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc638a6-ea14-434a-b9be-29d2c7c065c8 None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:02:19.757121 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc638a6-ea14-434a-b9be-29d2c7c065c8 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:02:19.757288 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc638a6-ea14-434a-b9be-29d2c7c065c8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:02:19.776319 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-bbc638a6-ea14-434a-b9be-29d2c7c065c8 None None] MFA Rules not processed for user `cc99c93da5074cf5a48bc2aa9a92c5e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 22:02:19.777611 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-bbc638a6-ea14-434a-b9be-29d2c7c065c8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:02:19.848679 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 614/1221] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jul 13 22:02:19 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 13 22:02:19.874696 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e95667-1e60-443c-a8d6-20abc95066dd None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:02:19.981390 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36e95667-1e60-443c-a8d6-20abc95066dd None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:02:20.018320 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309f30fd-6f8b-4af9-babe-bed09f87fb1a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:02:20.019189 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309f30fd-6f8b-4af9-babe-bed09f87fb1a None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:02:20.019440 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309f30fd-6f8b-4af9-babe-bed09f87fb1a None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:02:20.019710 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-309f30fd-6f8b-4af9-babe-bed09f87fb1a None admin] PATH_INFO: `/v3/roles/service` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:02:20.030127 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-309f30fd-6f8b-4af9-babe-bed09f87fb1a None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 22:02:20.037906 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application [None req-309f30fd-6f8b-4af9-babe-bed09f87fb1a None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 13 22:02:20.037906 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 22:02:20.037906 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 22:02:20.037906 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 22:02:20.037906 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 22:02:20.037906 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 22:02:20.037906 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 22:02:20.037906 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 22:02:20.037906 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 22:02:20.037906 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 22:02:20.037906 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 22:02:20.037906 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 22:02:20.037906 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 22:02:20.037906 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 22:02:20.037906 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 22:02:20.037906 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 22:02:20.039222 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 22:02:20.039222 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 22:02:20.039222 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 22:02:20.039222 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 22:02:20.039222 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 22:02:20.039222 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 22:02:20.039222 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 22:02:20.039222 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 22:02:20.039222 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 22:02:20.039222 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 22:02:20.039222 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 22:02:20.039222 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application with Lock( Jul 13 22:02:20.039222 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 22:02:20.039222 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._enter() Jul 13 22:02:20.039222 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 22:02:20.039222 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 22:02:20.040115 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 22:02:20.040115 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.creator() Jul 13 22:02:20.040115 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 22:02:20.040115 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application created_value = creator( Jul 13 22:02:20.040115 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 22:02:20.040115 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 22:02:20.040115 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 22:02:20.040115 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 22:02:20.040115 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 22:02:20.040115 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 22:02:20.040115 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 13 22:02:20.040115 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Jul 13 22:02:20.040115 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 608/1222] 10.222.0.52 () {56 vars in 1102 bytes} [Wed Jul 13 22:02:19 2022] GET /identity/v3/roles/service => generated 85 bytes in 165 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 22:02:20.045525 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc638a6-ea14-434a-b9be-29d2c7c065c8 None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:02:20.157901 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc638a6-ea14-434a-b9be-29d2c7c065c8 None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:02:20.185268 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:02:20.197866 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73c07b1-7cbc-41d4-87eb-b189a35e2927 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:02:20.198983 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73c07b1-7cbc-41d4-87eb-b189a35e2927 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:02:20.199321 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73c07b1-7cbc-41d4-87eb-b189a35e2927 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:02:20.199677 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d73c07b1-7cbc-41d4-87eb-b189a35e2927 None admin] PATH_INFO: `/v3/roles` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:02:20.200625 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d73c07b1-7cbc-41d4-87eb-b189a35e2927 None admin] RBAC: Adding query filter params (name=service) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 22:02:20.200940 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d73c07b1-7cbc-41d4-87eb-b189a35e2927 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:02:20.202537 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d73c07b1-7cbc-41d4-87eb-b189a35e2927 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:02:20.211859 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 615/1223] 10.222.0.52 () {56 vars in 1103 bytes} [Wed Jul 13 22:02:20 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 22:02:20.218372 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309f30fd-6f8b-4af9-babe-bed09f87fb1a None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:02:20.269474 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:02:20.335841 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-309f30fd-6f8b-4af9-babe-bed09f87fb1a None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:02:20.375706 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492bf415-3c4f-4b7c-8394-13d4369e62f3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:02:20.376543 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492bf415-3c4f-4b7c-8394-13d4369e62f3 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:02:20.376837 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492bf415-3c4f-4b7c-8394-13d4369e62f3 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:02:20.377112 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-492bf415-3c4f-4b7c-8394-13d4369e62f3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:02:20.389445 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-492bf415-3c4f-4b7c-8394-13d4369e62f3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:02:20.390959 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-492bf415-3c4f-4b7c-8394-13d4369e62f3 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:02:20.411453 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application [None req-492bf415-3c4f-4b7c-8394-13d4369e62f3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 22:02:20.411453 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 22:02:20.411453 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 22:02:20.411453 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 22:02:20.411453 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 22:02:20.411453 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 22:02:20.411453 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 22:02:20.411453 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 22:02:20.411453 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 22:02:20.411453 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 22:02:20.411453 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 22:02:20.411453 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 22:02:20.411453 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 22:02:20.411453 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 22:02:20.411453 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 22:02:20.412418 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 22:02:20.412418 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 22:02:20.412418 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 22:02:20.412418 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 22:02:20.412418 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 22:02:20.412418 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 22:02:20.412418 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 22:02:20.412418 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 22:02:20.412418 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application with Lock( Jul 13 22:02:20.412418 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 22:02:20.412418 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._enter() Jul 13 22:02:20.412418 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 22:02:20.412418 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 22:02:20.412418 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 22:02:20.412418 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.creator() Jul 13 22:02:20.412418 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 22:02:20.412418 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application created_value = creator( Jul 13 22:02:20.413331 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 22:02:20.413331 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 22:02:20.413331 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 22:02:20.413331 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Jul 13 22:02:20.413331 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 609/1224] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Jul 13 22:02:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 22:02:20.419194 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73c07b1-7cbc-41d4-87eb-b189a35e2927 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:02:20.527572 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d73c07b1-7cbc-41d4-87eb-b189a35e2927 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:02:20.567362 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8105e1-bab5-4413-b5fb-fe9c882d66cd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:02:20.568122 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8105e1-bab5-4413-b5fb-fe9c882d66cd None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:02:20.568405 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8105e1-bab5-4413-b5fb-fe9c882d66cd None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:02:20.568751 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f8105e1-bab5-4413-b5fb-fe9c882d66cd None admin] PATH_INFO: `/v3/domains` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:02:20.569520 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8105e1-bab5-4413-b5fb-fe9c882d66cd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 22:02:20.569927 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8105e1-bab5-4413-b5fb-fe9c882d66cd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:02:20.571502 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4f8105e1-bab5-4413-b5fb-fe9c882d66cd None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:02:20.585185 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 616/1225] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Jul 13 22:02:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 22:02:20.591954 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492bf415-3c4f-4b7c-8394-13d4369e62f3 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:02:20.695387 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:02:20.704227 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-492bf415-3c4f-4b7c-8394-13d4369e62f3 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:02:20.740365 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8826bcb7-667f-4b4f-bc75-5b5282b0e3a8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:02:20.741257 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8826bcb7-667f-4b4f-bc75-5b5282b0e3a8 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:02:20.741529 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8826bcb7-667f-4b4f-bc75-5b5282b0e3a8 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:02:20.741793 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8826bcb7-667f-4b4f-bc75-5b5282b0e3a8 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:02:20.750961 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8826bcb7-667f-4b4f-bc75-5b5282b0e3a8 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:02:20.752522 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8826bcb7-667f-4b4f-bc75-5b5282b0e3a8 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:02:20.769401 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application [None req-8826bcb7-667f-4b4f-bc75-5b5282b0e3a8 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 13 22:02:20.769401 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 22:02:20.769401 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 22:02:20.769401 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 22:02:20.769401 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 22:02:20.769401 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 22:02:20.769401 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 22:02:20.769401 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 22:02:20.769401 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 22:02:20.769401 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 22:02:20.769401 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 22:02:20.769401 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 22:02:20.769401 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 13 22:02:20.769401 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 13 22:02:20.769401 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 13 22:02:20.769401 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 13 22:02:20.770525 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 22:02:20.770525 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 22:02:20.770525 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 13 22:02:20.770525 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 22:02:20.770525 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 13 22:02:20.770525 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 22:02:20.770525 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 22:02:20.770525 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 22:02:20.770525 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 22:02:20.770525 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 22:02:20.770525 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 22:02:20.770525 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application with Lock( Jul 13 22:02:20.770525 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 22:02:20.770525 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._enter() Jul 13 22:02:20.770525 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 22:02:20.770525 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 22:02:20.770525 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 22:02:20.770525 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.creator() Jul 13 22:02:20.772099 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 22:02:20.772099 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application created_value = creator( Jul 13 22:02:20.772099 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 13 22:02:20.772099 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 13 22:02:20.772099 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 13 22:02:20.772099 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 13 22:02:20.772099 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 13 22:02:20.772099 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 13 22:02:20.772099 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 13 22:02:20.772099 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Jul 13 22:02:20.772099 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 610/1226] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Jul 13 22:02:20 2022] GET /identity/v3/users/placement => generated 87 bytes in 180 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 13 22:02:20.777646 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8105e1-bab5-4413-b5fb-fe9c882d66cd None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:02:20.828633 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:02:20.889038 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f8105e1-bab5-4413-b5fb-fe9c882d66cd None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:02:20.926403 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac021a0-d7e3-4c59-b228-7ba4c0280e29 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:02:20.927274 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac021a0-d7e3-4c59-b228-7ba4c0280e29 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:02:20.927607 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac021a0-d7e3-4c59-b228-7ba4c0280e29 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:02:20.927904 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac021a0-d7e3-4c59-b228-7ba4c0280e29 None admin] PATH_INFO: `/v3/users` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:02:20.928907 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac021a0-d7e3-4c59-b228-7ba4c0280e29 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 22:02:20.929217 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac021a0-d7e3-4c59-b228-7ba4c0280e29 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:02:20.931473 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac021a0-d7e3-4c59-b228-7ba4c0280e29 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:02:20.953618 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 617/1227] 10.222.0.52 () {56 vars in 1143 bytes} [Wed Jul 13 22:02:20 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 22:02:20.959826 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8826bcb7-667f-4b4f-bc75-5b5282b0e3a8 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:02:21.071363 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8826bcb7-667f-4b4f-bc75-5b5282b0e3a8 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:02:21.107922 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be6b32b-7f5b-4b77-b3bd-ae11c873316f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:02:21.108724 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be6b32b-7f5b-4b77-b3bd-ae11c873316f None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:02:21.108998 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be6b32b-7f5b-4b77-b3bd-ae11c873316f None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:02:21.109266 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be6b32b-7f5b-4b77-b3bd-ae11c873316f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:02:21.120007 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be6b32b-7f5b-4b77-b3bd-ae11c873316f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:02:21.121459 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be6b32b-7f5b-4b77-b3bd-ae11c873316f None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:02:21.139147 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application [None req-7be6b32b-7f5b-4b77-b3bd-ae11c873316f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 22:02:21.139147 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 22:02:21.139147 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 22:02:21.139147 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 22:02:21.139147 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 22:02:21.139147 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 22:02:21.139147 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 22:02:21.139147 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 22:02:21.139147 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 22:02:21.139147 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 22:02:21.139147 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 22:02:21.139147 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 22:02:21.139147 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 22:02:21.139147 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 22:02:21.139147 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 22:02:21.140268 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 22:02:21.140268 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 22:02:21.140268 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 22:02:21.140268 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 22:02:21.140268 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 22:02:21.140268 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 22:02:21.140268 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 22:02:21.140268 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 22:02:21.140268 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application with Lock( Jul 13 22:02:21.140268 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 22:02:21.140268 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._enter() Jul 13 22:02:21.140268 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 22:02:21.140268 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 22:02:21.140268 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 22:02:21.140268 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.creator() Jul 13 22:02:21.140268 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 22:02:21.140268 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application created_value = creator( Jul 13 22:02:21.141529 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 22:02:21.141529 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 22:02:21.141529 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 22:02:21.141529 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Jul 13 22:02:21.141529 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 611/1228] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Jul 13 22:02:20 2022] GET /identity/v3/domains/Default => generated 87 bytes in 181 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 13 22:02:21.146632 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac021a0-d7e3-4c59-b228-7ba4c0280e29 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:02:21.251490 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac021a0-d7e3-4c59-b228-7ba4c0280e29 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:02:21.253664 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:02:21.287682 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a1699f-971f-4c87-86f2-6c51238f057c None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:02:21.288354 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a1699f-971f-4c87-86f2-6c51238f057c None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:02:21.288667 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a1699f-971f-4c87-86f2-6c51238f057c None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:02:21.289166 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2a1699f-971f-4c87-86f2-6c51238f057c None admin] PATH_INFO: `/v3/domains` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:02:21.290250 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a1699f-971f-4c87-86f2-6c51238f057c None admin] RBAC: Adding query filter params (name=Default) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 22:02:21.290557 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a1699f-971f-4c87-86f2-6c51238f057c None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:02:21.292045 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2a1699f-971f-4c87-86f2-6c51238f057c None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:02:21.305022 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 618/1229] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Jul 13 22:02:21 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 22:02:21.311773 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be6b32b-7f5b-4b77-b3bd-ae11c873316f None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:02:21.323852 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:02:21.424683 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be6b32b-7f5b-4b77-b3bd-ae11c873316f None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:02:21.461702 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0deda181-b51f-45ba-8022-ee578ee2e10a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:02:21.462566 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0deda181-b51f-45ba-8022-ee578ee2e10a None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:02:21.462849 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0deda181-b51f-45ba-8022-ee578ee2e10a None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:02:21.463129 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0deda181-b51f-45ba-8022-ee578ee2e10a None admin] PATH_INFO: `/v3/projects/service` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:02:21.470869 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0deda181-b51f-45ba-8022-ee578ee2e10a None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:02:21.472426 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0deda181-b51f-45ba-8022-ee578ee2e10a None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:02:21.487338 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application [None req-0deda181-b51f-45ba-8022-ee578ee2e10a None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 13 22:02:21.487338 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 22:02:21.487338 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 22:02:21.487338 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 22:02:21.487338 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 22:02:21.487338 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 22:02:21.487338 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 22:02:21.487338 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 22:02:21.487338 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 22:02:21.487338 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 22:02:21.487338 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 22:02:21.487338 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 22:02:21.487338 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 13 22:02:21.487338 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 13 22:02:21.487338 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 13 22:02:21.488362 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 13 22:02:21.488362 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 22:02:21.488362 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 22:02:21.488362 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 22:02:21.488362 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 22:02:21.488362 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 22:02:21.488362 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 22:02:21.488362 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 22:02:21.488362 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application with Lock( Jul 13 22:02:21.488362 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 22:02:21.488362 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._enter() Jul 13 22:02:21.488362 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 22:02:21.488362 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 22:02:21.488362 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 22:02:21.488362 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.creator() Jul 13 22:02:21.488362 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 22:02:21.488362 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application created_value = creator( Jul 13 22:02:21.489317 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 13 22:02:21.489317 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 13 22:02:21.489317 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 13 22:02:21.489317 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 13 22:02:21.489317 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 13 22:02:21.489317 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 13 22:02:21.489317 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 13 22:02:21.489317 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Jul 13 22:02:21.489317 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 612/1230] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Jul 13 22:02:21 2022] GET /identity/v3/projects/service => generated 88 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 22:02:21.496157 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a1699f-971f-4c87-86f2-6c51238f057c None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:02:21.613215 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2a1699f-971f-4c87-86f2-6c51238f057c None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:02:21.650510 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2797715-8dbb-4f2a-b2dd-c0149511dc55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:02:21.651205 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2797715-8dbb-4f2a-b2dd-c0149511dc55 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:02:21.651520 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2797715-8dbb-4f2a-b2dd-c0149511dc55 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:02:21.651759 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2797715-8dbb-4f2a-b2dd-c0149511dc55 None admin] PATH_INFO: `/v3/projects` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:02:21.652556 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2797715-8dbb-4f2a-b2dd-c0149511dc55 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 22:02:21.652855 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2797715-8dbb-4f2a-b2dd-c0149511dc55 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:02:21.655168 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2797715-8dbb-4f2a-b2dd-c0149511dc55 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:02:21.670949 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 619/1231] 10.222.0.52 () {56 vars in 1148 bytes} [Wed Jul 13 22:02:21 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 22:02:21.676981 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0deda181-b51f-45ba-8022-ee578ee2e10a None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:02:21.754361 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:02:21.793034 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0deda181-b51f-45ba-8022-ee578ee2e10a None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:02:21.835536 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4b9a7dfb-52f6-4b5f-8c4d-2ad85073b39a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:02:21.837189 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9a7dfb-52f6-4b5f-8c4d-2ad85073b39a None admin] REQUEST_METHOD: `PUT` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:02:21.837476 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9a7dfb-52f6-4b5f-8c4d-2ad85073b39a None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:02:21.837779 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4b9a7dfb-52f6-4b5f-8c4d-2ad85073b39a None admin] PATH_INFO: `/v3/projects/417e3e2f1333481795a089bef2e24bea/users/ac673d641d8849c79bf4cf71c9805587/roles/70c34ea493484322911dab62b4a56e75` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:02:21.856039 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b9a7dfb-52f6-4b5f-8c4d-2ad85073b39a None admin] RBAC: Authorizing `identity:create_grant(project_id=417e3e2f1333481795a089bef2e24bea, user_id=ac673d641d8849c79bf4cf71c9805587, role_id=70c34ea493484322911dab62b4a56e75)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:02:21.857571 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4b9a7dfb-52f6-4b5f-8c4d-2ad85073b39a None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:02:21.873272 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 613/1232] 10.222.0.52 () {58 vars in 1439 bytes} [Wed Jul 13 22:02:21 2022] PUT /identity/v3/projects/417e3e2f1333481795a089bef2e24bea/users/ac673d641d8849c79bf4cf71c9805587/roles/70c34ea493484322911dab62b4a56e75 => generated 0 bytes in 197 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 13 22:02:24.040131 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cb13b6-c7fc-48eb-b2ee-b78b7c006df7 None None] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:02:24.040131 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cb13b6-c7fc-48eb-b2ee-b78b7c006df7 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:02:24.040131 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cb13b6-c7fc-48eb-b2ee-b78b7c006df7 None None] PATH_INFO: `/` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:02:24.041038 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 620/1233] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jul 13 22:02:24 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 22:02:24.049109 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278ad1d8-337b-4ff7-a421-680518ed0564 None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:02:24.049541 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278ad1d8-337b-4ff7-a421-680518ed0564 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:02:24.049699 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-278ad1d8-337b-4ff7-a421-680518ed0564 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:02:24.067008 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-278ad1d8-337b-4ff7-a421-680518ed0564 None None] MFA Rules not processed for user `cc99c93da5074cf5a48bc2aa9a92c5e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 22:02:24.068213 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-278ad1d8-337b-4ff7-a421-680518ed0564 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:02:24.151500 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 614/1234] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jul 13 22:02:24 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 13 22:02:24.180015 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9cb13b6-c7fc-48eb-b2ee-b78b7c006df7 None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:02:24.217108 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:02:24.306790 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9cb13b6-c7fc-48eb-b2ee-b78b7c006df7 None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:02:24.353768 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2828bbf5-95d5-4fe6-9c65-0ac8d135b3ed None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:02:24.359021 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2828bbf5-95d5-4fe6-9c65-0ac8d135b3ed None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:02:24.359490 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2828bbf5-95d5-4fe6-9c65-0ac8d135b3ed None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:02:24.359803 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2828bbf5-95d5-4fe6-9c65-0ac8d135b3ed None admin] PATH_INFO: `/v3/roles/service` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:02:24.379587 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2828bbf5-95d5-4fe6-9c65-0ac8d135b3ed None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 22:02:24.389046 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application [None req-2828bbf5-95d5-4fe6-9c65-0ac8d135b3ed None admin] Could not find role: service.: keystone.exception.RoleNotFound: Could not find role: service. Jul 13 22:02:24.389046 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 22:02:24.389046 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 22:02:24.389046 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 22:02:24.389046 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 22:02:24.389046 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 22:02:24.389046 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 22:02:24.389046 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 22:02:24.389046 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 22:02:24.389046 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 22:02:24.389046 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 22:02:24.389046 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 22:02:24.389046 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 22:02:24.389046 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 22:02:24.389046 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 22:02:24.389046 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 22:02:24.391345 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 22:02:24.391345 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 22:02:24.391345 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 22:02:24.391345 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 22:02:24.391345 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 22:02:24.391345 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 22:02:24.391345 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 22:02:24.391345 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 22:02:24.391345 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 22:02:24.391345 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 22:02:24.391345 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 22:02:24.391345 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application with Lock( Jul 13 22:02:24.391345 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 22:02:24.391345 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._enter() Jul 13 22:02:24.391345 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 22:02:24.391345 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 22:02:24.392828 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 22:02:24.392828 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.creator() Jul 13 22:02:24.392828 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 22:02:24.392828 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application created_value = creator( Jul 13 22:02:24.392828 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 22:02:24.392828 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 22:02:24.392828 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 22:02:24.392828 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 22:02:24.392828 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 22:02:24.392828 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 22:02:24.392828 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: service. Jul 13 22:02:24.392828 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Jul 13 22:02:24.392828 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 621/1235] 10.222.0.52 () {56 vars in 1102 bytes} [Wed Jul 13 22:02:24 2022] GET /identity/v3/roles/service => generated 85 bytes in 211 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 22:02:24.397238 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278ad1d8-337b-4ff7-a421-680518ed0564 None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:02:24.459579 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:02:24.539070 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-278ad1d8-337b-4ff7-a421-680518ed0564 None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:02:24.583385 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4e0ab1-1536-4a47-8457-2a06e3b36555 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:02:24.583996 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4e0ab1-1536-4a47-8457-2a06e3b36555 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:02:24.584276 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4e0ab1-1536-4a47-8457-2a06e3b36555 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:02:24.584548 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4e0ab1-1536-4a47-8457-2a06e3b36555 None admin] PATH_INFO: `/v3/roles` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:02:24.585315 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4e0ab1-1536-4a47-8457-2a06e3b36555 None admin] RBAC: Adding query filter params (name=service) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 22:02:24.585696 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4e0ab1-1536-4a47-8457-2a06e3b36555 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:02:24.587267 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4e0ab1-1536-4a47-8457-2a06e3b36555 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:02:24.600434 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 615/1236] 10.222.0.52 () {56 vars in 1103 bytes} [Wed Jul 13 22:02:24 2022] GET /identity/v3/roles?name=service => generated 326 bytes in 199 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 22:02:24.602024 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2828bbf5-95d5-4fe6-9c65-0ac8d135b3ed None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:02:24.692654 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:02:24.708664 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2828bbf5-95d5-4fe6-9c65-0ac8d135b3ed None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:02:24.742563 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7afe39-b099-4863-ad98-a8db67f0f3f4 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:02:24.743383 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7afe39-b099-4863-ad98-a8db67f0f3f4 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:02:24.743657 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7afe39-b099-4863-ad98-a8db67f0f3f4 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:02:24.743960 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf7afe39-b099-4863-ad98-a8db67f0f3f4 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:02:24.754432 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7afe39-b099-4863-ad98-a8db67f0f3f4 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:02:24.755930 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf7afe39-b099-4863-ad98-a8db67f0f3f4 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:02:24.773476 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application [None req-bf7afe39-b099-4863-ad98-a8db67f0f3f4 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 22:02:24.773476 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 22:02:24.773476 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 22:02:24.773476 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 22:02:24.773476 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 22:02:24.773476 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 22:02:24.773476 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 22:02:24.773476 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 22:02:24.773476 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 22:02:24.773476 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 22:02:24.773476 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 22:02:24.773476 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 22:02:24.773476 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 22:02:24.773476 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 22:02:24.773476 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 22:02:24.774575 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 22:02:24.774575 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 22:02:24.774575 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 22:02:24.774575 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 22:02:24.774575 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 22:02:24.774575 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 22:02:24.774575 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 22:02:24.774575 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 22:02:24.774575 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application with Lock( Jul 13 22:02:24.774575 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 22:02:24.774575 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._enter() Jul 13 22:02:24.774575 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 22:02:24.774575 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 22:02:24.774575 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 22:02:24.774575 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.creator() Jul 13 22:02:24.774575 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 22:02:24.774575 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application created_value = creator( Jul 13 22:02:24.775546 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 22:02:24.775546 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 22:02:24.775546 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 22:02:24.775546 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Jul 13 22:02:24.775546 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 622/1237] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Jul 13 22:02:24 2022] GET /identity/v3/domains/Default => generated 87 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 22:02:24.781241 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4e0ab1-1536-4a47-8457-2a06e3b36555 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:02:24.902648 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4e0ab1-1536-4a47-8457-2a06e3b36555 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:02:24.914196 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:02:24.941692 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c17f0a-20f2-4e82-ab5f-fad2677057c7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:02:24.942360 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c17f0a-20f2-4e82-ab5f-fad2677057c7 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:02:24.942763 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c17f0a-20f2-4e82-ab5f-fad2677057c7 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:02:24.943035 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c17f0a-20f2-4e82-ab5f-fad2677057c7 None admin] PATH_INFO: `/v3/domains` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:02:24.943754 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c17f0a-20f2-4e82-ab5f-fad2677057c7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 22:02:24.944058 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c17f0a-20f2-4e82-ab5f-fad2677057c7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:02:24.945955 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7c17f0a-20f2-4e82-ab5f-fad2677057c7 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:02:24.964742 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 616/1238] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Jul 13 22:02:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 179 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 22:02:24.965990 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7afe39-b099-4863-ad98-a8db67f0f3f4 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:02:25.068530 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf7afe39-b099-4863-ad98-a8db67f0f3f4 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:02:25.104970 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9443ab29-8197-42f4-9fdd-6f9fdf548b2a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:02:25.105766 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9443ab29-8197-42f4-9fdd-6f9fdf548b2a None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:02:25.106044 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9443ab29-8197-42f4-9fdd-6f9fdf548b2a None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:02:25.106315 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9443ab29-8197-42f4-9fdd-6f9fdf548b2a None admin] PATH_INFO: `/v3/users/placement` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:02:25.109225 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:02:25.114851 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9443ab29-8197-42f4-9fdd-6f9fdf548b2a None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:02:25.116483 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9443ab29-8197-42f4-9fdd-6f9fdf548b2a None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:02:25.132582 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application [None req-9443ab29-8197-42f4-9fdd-6f9fdf548b2a None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 13 22:02:25.132582 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 22:02:25.132582 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 22:02:25.132582 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 22:02:25.132582 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 22:02:25.132582 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 22:02:25.132582 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 22:02:25.132582 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 22:02:25.132582 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 22:02:25.132582 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 22:02:25.132582 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 22:02:25.132582 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 22:02:25.132582 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 13 22:02:25.132582 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 13 22:02:25.132582 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 13 22:02:25.132582 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 13 22:02:25.133931 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 22:02:25.133931 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 22:02:25.133931 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 13 22:02:25.133931 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 22:02:25.133931 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 13 22:02:25.133931 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 22:02:25.133931 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 22:02:25.133931 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 22:02:25.133931 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 22:02:25.133931 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 22:02:25.133931 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 22:02:25.133931 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application with Lock( Jul 13 22:02:25.133931 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 22:02:25.133931 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._enter() Jul 13 22:02:25.133931 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 22:02:25.133931 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 22:02:25.133931 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 22:02:25.133931 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.creator() Jul 13 22:02:25.134986 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 22:02:25.134986 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application created_value = creator( Jul 13 22:02:25.134986 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 13 22:02:25.134986 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 13 22:02:25.134986 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 13 22:02:25.134986 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 13 22:02:25.134986 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 13 22:02:25.134986 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 13 22:02:25.134986 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 13 22:02:25.134986 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Jul 13 22:02:25.134986 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 623/1239] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Jul 13 22:02:24 2022] GET /identity/v3/users/placement => generated 87 bytes in 169 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 22:02:25.140380 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c17f0a-20f2-4e82-ab5f-fad2677057c7 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:02:25.244948 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7c17f0a-20f2-4e82-ab5f-fad2677057c7 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:02:25.280012 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4963d6-fbae-4831-9ea9-54861ffde57b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:02:25.280790 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4963d6-fbae-4831-9ea9-54861ffde57b None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:02:25.281163 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4963d6-fbae-4831-9ea9-54861ffde57b None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:02:25.281464 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc4963d6-fbae-4831-9ea9-54861ffde57b None admin] PATH_INFO: `/v3/users` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:02:25.282310 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4963d6-fbae-4831-9ea9-54861ffde57b None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 22:02:25.282669 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4963d6-fbae-4831-9ea9-54861ffde57b None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:02:25.284255 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc4963d6-fbae-4831-9ea9-54861ffde57b None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:02:25.302414 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 617/1240] 10.222.0.52 () {56 vars in 1143 bytes} [Wed Jul 13 22:02:25 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 22:02:25.309027 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9443ab29-8197-42f4-9fdd-6f9fdf548b2a None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:02:25.433510 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9443ab29-8197-42f4-9fdd-6f9fdf548b2a None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:02:25.468695 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42814974-2457-4ae6-bf25-1a2b7c2c2b13 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:02:25.469358 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42814974-2457-4ae6-bf25-1a2b7c2c2b13 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:02:25.469644 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42814974-2457-4ae6-bf25-1a2b7c2c2b13 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:02:25.469921 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-42814974-2457-4ae6-bf25-1a2b7c2c2b13 None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:02:25.534339 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:02:25.541794 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42814974-2457-4ae6-bf25-1a2b7c2c2b13 None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:02:25.543311 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-42814974-2457-4ae6-bf25-1a2b7c2c2b13 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:02:25.647406 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 624/1241] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Jul 13 22:02:25 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 13 22:02:25.655453 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4963d6-fbae-4831-9ea9-54861ffde57b None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:02:25.686734 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:02:25.774824 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc4963d6-fbae-4831-9ea9-54861ffde57b None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:02:25.810059 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712be9c8-a0fd-46f3-bbfb-7bf68559498f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:02:25.810892 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712be9c8-a0fd-46f3-bbfb-7bf68559498f None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:02:25.811298 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712be9c8-a0fd-46f3-bbfb-7bf68559498f None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:02:25.811601 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712be9c8-a0fd-46f3-bbfb-7bf68559498f None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:02:25.825356 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712be9c8-a0fd-46f3-bbfb-7bf68559498f None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:02:25.827077 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712be9c8-a0fd-46f3-bbfb-7bf68559498f None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:02:25.847364 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application [None req-712be9c8-a0fd-46f3-bbfb-7bf68559498f None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 22:02:25.847364 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 22:02:25.847364 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 22:02:25.847364 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 22:02:25.847364 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 22:02:25.847364 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 22:02:25.847364 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 22:02:25.847364 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 22:02:25.847364 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 22:02:25.847364 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 22:02:25.847364 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 22:02:25.847364 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 22:02:25.847364 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 22:02:25.847364 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 22:02:25.847364 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 22:02:25.848396 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 22:02:25.848396 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 22:02:25.848396 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 22:02:25.848396 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 22:02:25.848396 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 22:02:25.848396 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 22:02:25.848396 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 22:02:25.848396 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 22:02:25.848396 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application with Lock( Jul 13 22:02:25.848396 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 22:02:25.848396 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._enter() Jul 13 22:02:25.848396 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 22:02:25.848396 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 22:02:25.848396 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 22:02:25.848396 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.creator() Jul 13 22:02:25.848396 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 22:02:25.848396 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application created_value = creator( Jul 13 22:02:25.849302 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 22:02:25.849302 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 22:02:25.849302 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 22:02:25.849302 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Jul 13 22:02:25.849302 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 618/1242] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Jul 13 22:02:25 2022] GET /identity/v3/domains/Default => generated 87 bytes in 194 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 22:02:25.856051 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42814974-2457-4ae6-bf25-1a2b7c2c2b13 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:02:25.960571 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:02:25.968841 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-42814974-2457-4ae6-bf25-1a2b7c2c2b13 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:02:26.008677 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba0020a-6b0c-4025-a647-2ec8e0c0fcb2 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:02:26.009619 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba0020a-6b0c-4025-a647-2ec8e0c0fcb2 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:02:26.009906 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba0020a-6b0c-4025-a647-2ec8e0c0fcb2 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:02:26.010216 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba0020a-6b0c-4025-a647-2ec8e0c0fcb2 None admin] PATH_INFO: `/v3/domains` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:02:26.011029 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba0020a-6b0c-4025-a647-2ec8e0c0fcb2 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 22:02:26.011335 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba0020a-6b0c-4025-a647-2ec8e0c0fcb2 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:02:26.012732 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba0020a-6b0c-4025-a647-2ec8e0c0fcb2 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:02:26.030894 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 625/1243] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Jul 13 22:02:25 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 22:02:26.037040 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712be9c8-a0fd-46f3-bbfb-7bf68559498f None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:02:26.130743 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:02:26.157846 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712be9c8-a0fd-46f3-bbfb-7bf68559498f None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:02:26.200233 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c6de1b-db41-4e7a-9a33-04eb49afb5e7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:02:26.201027 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c6de1b-db41-4e7a-9a33-04eb49afb5e7 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:02:26.201303 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c6de1b-db41-4e7a-9a33-04eb49afb5e7 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:02:26.201575 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53c6de1b-db41-4e7a-9a33-04eb49afb5e7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:02:26.208549 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c6de1b-db41-4e7a-9a33-04eb49afb5e7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:02:26.210054 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53c6de1b-db41-4e7a-9a33-04eb49afb5e7 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:02:26.225826 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application [None req-53c6de1b-db41-4e7a-9a33-04eb49afb5e7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 13 22:02:26.225826 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 22:02:26.225826 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 22:02:26.225826 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 22:02:26.225826 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 22:02:26.225826 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 22:02:26.225826 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 22:02:26.225826 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 22:02:26.225826 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 22:02:26.225826 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 22:02:26.225826 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 22:02:26.225826 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 22:02:26.225826 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 13 22:02:26.225826 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 13 22:02:26.225826 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 13 22:02:26.227120 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 13 22:02:26.227120 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 22:02:26.227120 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 22:02:26.227120 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 22:02:26.227120 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 22:02:26.227120 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 22:02:26.227120 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 22:02:26.227120 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 22:02:26.227120 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application with Lock( Jul 13 22:02:26.227120 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 22:02:26.227120 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._enter() Jul 13 22:02:26.227120 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 22:02:26.227120 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 22:02:26.227120 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 22:02:26.227120 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.creator() Jul 13 22:02:26.227120 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 22:02:26.227120 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application created_value = creator( Jul 13 22:02:26.228033 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 13 22:02:26.228033 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 13 22:02:26.228033 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 13 22:02:26.228033 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 13 22:02:26.228033 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 13 22:02:26.228033 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 13 22:02:26.228033 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 13 22:02:26.228033 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Jul 13 22:02:26.228033 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 619/1244] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Jul 13 22:02:26 2022] GET /identity/v3/projects/service => generated 88 bytes in 191 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 22:02:26.233425 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba0020a-6b0c-4025-a647-2ec8e0c0fcb2 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:02:26.350653 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba0020a-6b0c-4025-a647-2ec8e0c0fcb2 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:02:26.394774 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5433e260-118b-4b32-ba12-7bb040da0834 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:02:26.395348 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5433e260-118b-4b32-ba12-7bb040da0834 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:02:26.395623 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5433e260-118b-4b32-ba12-7bb040da0834 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:02:26.395894 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5433e260-118b-4b32-ba12-7bb040da0834 None admin] PATH_INFO: `/v3/projects` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:02:26.396741 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5433e260-118b-4b32-ba12-7bb040da0834 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 22:02:26.397226 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5433e260-118b-4b32-ba12-7bb040da0834 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:02:26.398706 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5433e260-118b-4b32-ba12-7bb040da0834 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:02:26.416087 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 626/1245] 10.222.0.52 () {56 vars in 1148 bytes} [Wed Jul 13 22:02:26 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 178 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 22:02:26.417780 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c6de1b-db41-4e7a-9a33-04eb49afb5e7 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:02:26.534170 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53c6de1b-db41-4e7a-9a33-04eb49afb5e7 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:02:26.574910 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ef752c-4e1d-44e3-b0ae-52e99496fcaa None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:02:26.575580 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ef752c-4e1d-44e3-b0ae-52e99496fcaa None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:02:26.575936 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ef752c-4e1d-44e3-b0ae-52e99496fcaa None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:02:26.576289 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ef752c-4e1d-44e3-b0ae-52e99496fcaa None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:02:26.577254 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ef752c-4e1d-44e3-b0ae-52e99496fcaa None admin] RBAC: Adding query filter params (role.id=70c34ea493484322911dab62b4a56e75, scope.project.id=417e3e2f1333481795a089bef2e24bea, user.id=ac673d641d8849c79bf4cf71c9805587) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 22:02:26.577627 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ef752c-4e1d-44e3-b0ae-52e99496fcaa None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:02:26.579453 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ef752c-4e1d-44e3-b0ae-52e99496fcaa None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:02:26.586842 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 620/1246] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Jul 13 22:02:26 2022] GET /identity/v3/role_assignments?user.id=ac673d641d8849c79bf4cf71c9805587&scope.project.id=417e3e2f1333481795a089bef2e24bea&role.id=70c34ea493484322911dab62b4a56e75 => generated 608 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 22:02:28.856469 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f188a044-a167-4284-9afd-498ad1ba09de None None] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:02:28.857003 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f188a044-a167-4284-9afd-498ad1ba09de None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:02:28.857003 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f188a044-a167-4284-9afd-498ad1ba09de None None] PATH_INFO: `/` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:02:28.858448 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 627/1247] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jul 13 22:02:28 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 22:02:28.865392 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc20f09-8d8d-4276-b31d-eacc4494e225 None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:02:28.865885 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc20f09-8d8d-4276-b31d-eacc4494e225 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:02:28.866055 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bc20f09-8d8d-4276-b31d-eacc4494e225 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:02:28.871733 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:02:28.886051 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-1bc20f09-8d8d-4276-b31d-eacc4494e225 None None] MFA Rules not processed for user `cc99c93da5074cf5a48bc2aa9a92c5e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 22:02:28.887510 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-1bc20f09-8d8d-4276-b31d-eacc4494e225 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:02:28.966669 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 621/1248] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jul 13 22:02:28 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 13 22:02:28.990523 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f188a044-a167-4284-9afd-498ad1ba09de None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:02:29.010136 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:02:29.099536 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f188a044-a167-4284-9afd-498ad1ba09de None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:02:29.136533 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f7190a-3207-4df2-99ce-2c791c5325c0 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:02:29.137231 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f7190a-3207-4df2-99ce-2c791c5325c0 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:02:29.137518 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f7190a-3207-4df2-99ce-2c791c5325c0 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:02:29.137882 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28f7190a-3207-4df2-99ce-2c791c5325c0 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:02:29.150335 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28f7190a-3207-4df2-99ce-2c791c5325c0 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 22:02:29.159557 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application [None req-28f7190a-3207-4df2-99ce-2c791c5325c0 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 13 22:02:29.159557 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 22:02:29.159557 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 22:02:29.159557 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 22:02:29.159557 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 22:02:29.159557 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 22:02:29.159557 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 22:02:29.159557 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 22:02:29.159557 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 22:02:29.159557 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 22:02:29.159557 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 22:02:29.159557 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 22:02:29.159557 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 22:02:29.159557 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 22:02:29.159557 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 22:02:29.159557 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 22:02:29.160514 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 22:02:29.160514 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 22:02:29.160514 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 22:02:29.160514 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 22:02:29.160514 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 22:02:29.160514 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 22:02:29.160514 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 22:02:29.160514 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 22:02:29.160514 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 22:02:29.160514 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 22:02:29.160514 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 22:02:29.160514 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application with Lock( Jul 13 22:02:29.160514 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 22:02:29.160514 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._enter() Jul 13 22:02:29.160514 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 22:02:29.160514 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 22:02:29.161365 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 22:02:29.161365 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.creator() Jul 13 22:02:29.161365 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 22:02:29.161365 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application created_value = creator( Jul 13 22:02:29.161365 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 22:02:29.161365 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 22:02:29.161365 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 22:02:29.161365 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 22:02:29.161365 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 22:02:29.161365 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 22:02:29.161365 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 13 22:02:29.161365 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Jul 13 22:02:29.161365 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 628/1249] 10.222.0.52 () {56 vars in 1096 bytes} [Wed Jul 13 22:02:28 2022] GET /identity/v3/roles/admin => generated 83 bytes in 170 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 22:02:29.166026 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc20f09-8d8d-4276-b31d-eacc4494e225 None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:02:29.272472 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bc20f09-8d8d-4276-b31d-eacc4494e225 None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:02:29.303688 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:02:29.309582 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317e3962-c086-473e-b044-9febe2137e44 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:02:29.310254 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317e3962-c086-473e-b044-9febe2137e44 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:02:29.311061 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317e3962-c086-473e-b044-9febe2137e44 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:02:29.311355 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-317e3962-c086-473e-b044-9febe2137e44 None admin] PATH_INFO: `/v3/roles` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:02:29.312208 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-317e3962-c086-473e-b044-9febe2137e44 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 22:02:29.312496 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-317e3962-c086-473e-b044-9febe2137e44 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:02:29.313928 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-317e3962-c086-473e-b044-9febe2137e44 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:02:29.324192 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 622/1250] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Jul 13 22:02:29 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 22:02:29.330421 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f7190a-3207-4df2-99ce-2c791c5325c0 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:02:29.419271 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:02:29.447594 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28f7190a-3207-4df2-99ce-2c791c5325c0 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:02:29.486159 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9018dc1a-ff31-4656-a0bd-658181c96010 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:02:29.487033 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9018dc1a-ff31-4656-a0bd-658181c96010 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:02:29.487315 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9018dc1a-ff31-4656-a0bd-658181c96010 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:02:29.487549 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9018dc1a-ff31-4656-a0bd-658181c96010 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:02:29.500304 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9018dc1a-ff31-4656-a0bd-658181c96010 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:02:29.501971 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9018dc1a-ff31-4656-a0bd-658181c96010 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:02:29.520807 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application [None req-9018dc1a-ff31-4656-a0bd-658181c96010 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 22:02:29.520807 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 22:02:29.520807 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 22:02:29.520807 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 22:02:29.520807 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 22:02:29.520807 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 22:02:29.520807 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 22:02:29.520807 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 22:02:29.520807 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 22:02:29.520807 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 22:02:29.520807 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 22:02:29.520807 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 22:02:29.520807 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 22:02:29.520807 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 22:02:29.520807 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 22:02:29.522710 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 22:02:29.522710 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 22:02:29.522710 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 22:02:29.522710 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 22:02:29.522710 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 22:02:29.522710 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 22:02:29.522710 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 22:02:29.522710 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 22:02:29.522710 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application with Lock( Jul 13 22:02:29.522710 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 22:02:29.522710 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._enter() Jul 13 22:02:29.522710 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 22:02:29.522710 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 22:02:29.522710 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 22:02:29.522710 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.creator() Jul 13 22:02:29.522710 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 22:02:29.522710 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application created_value = creator( Jul 13 22:02:29.523774 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 22:02:29.523774 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 22:02:29.523774 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 22:02:29.523774 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Jul 13 22:02:29.523774 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 629/1251] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Jul 13 22:02:29 2022] GET /identity/v3/domains/Default => generated 87 bytes in 192 msecs (HTTP/1.1 404) 5 headers in 183 bytes (2 switches on core 0) Jul 13 22:02:29.528140 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317e3962-c086-473e-b044-9febe2137e44 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:02:29.640056 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-317e3962-c086-473e-b044-9febe2137e44 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:02:29.682404 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d906e0-cde9-4464-871e-bfc3efc234c1 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:02:29.682999 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d906e0-cde9-4464-871e-bfc3efc234c1 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:02:29.683280 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d906e0-cde9-4464-871e-bfc3efc234c1 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:02:29.683555 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d906e0-cde9-4464-871e-bfc3efc234c1 None admin] PATH_INFO: `/v3/domains` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:02:29.684453 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d906e0-cde9-4464-871e-bfc3efc234c1 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 22:02:29.684737 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d906e0-cde9-4464-871e-bfc3efc234c1 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:02:29.686215 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d906e0-cde9-4464-871e-bfc3efc234c1 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:02:29.699899 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 623/1252] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Jul 13 22:02:29 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 172 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 22:02:29.706059 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9018dc1a-ff31-4656-a0bd-658181c96010 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:02:29.820797 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9018dc1a-ff31-4656-a0bd-658181c96010 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:02:29.849207 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:02:29.857812 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5852c9-0469-46c6-a81a-3b6c28f26686 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:02:29.858670 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5852c9-0469-46c6-a81a-3b6c28f26686 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:02:29.858949 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5852c9-0469-46c6-a81a-3b6c28f26686 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:02:29.859223 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd5852c9-0469-46c6-a81a-3b6c28f26686 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:02:29.867870 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5852c9-0469-46c6-a81a-3b6c28f26686 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:02:29.869351 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd5852c9-0469-46c6-a81a-3b6c28f26686 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:02:29.886012 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application [None req-dd5852c9-0469-46c6-a81a-3b6c28f26686 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 13 22:02:29.886012 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 22:02:29.886012 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 22:02:29.886012 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 22:02:29.886012 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 22:02:29.886012 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 22:02:29.886012 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 22:02:29.886012 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 22:02:29.886012 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 22:02:29.886012 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 22:02:29.886012 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 22:02:29.886012 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 22:02:29.886012 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 13 22:02:29.886012 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 13 22:02:29.886012 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 13 22:02:29.886012 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 13 22:02:29.887296 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 22:02:29.887296 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 22:02:29.887296 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 13 22:02:29.887296 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 22:02:29.887296 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 13 22:02:29.887296 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 22:02:29.887296 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 22:02:29.887296 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 22:02:29.887296 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 22:02:29.887296 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 22:02:29.887296 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 22:02:29.887296 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application with Lock( Jul 13 22:02:29.887296 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 22:02:29.887296 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._enter() Jul 13 22:02:29.887296 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 22:02:29.887296 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 22:02:29.887296 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 22:02:29.887296 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.creator() Jul 13 22:02:29.888493 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 22:02:29.888493 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application created_value = creator( Jul 13 22:02:29.888493 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 13 22:02:29.888493 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 13 22:02:29.888493 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 13 22:02:29.888493 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 13 22:02:29.888493 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 13 22:02:29.888493 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 13 22:02:29.888493 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 13 22:02:29.888493 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Jul 13 22:02:29.892634 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 630/1253] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Jul 13 22:02:29 2022] GET /identity/v3/users/placement => generated 87 bytes in 187 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 22:02:29.893631 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d906e0-cde9-4464-871e-bfc3efc234c1 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:02:29.963135 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:02:30.012918 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d906e0-cde9-4464-871e-bfc3efc234c1 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:02:30.049970 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1fa56e-6e5c-48ce-8065-d98c2266f5b3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:02:30.050689 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1fa56e-6e5c-48ce-8065-d98c2266f5b3 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:02:30.050970 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1fa56e-6e5c-48ce-8065-d98c2266f5b3 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:02:30.051246 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad1fa56e-6e5c-48ce-8065-d98c2266f5b3 None admin] PATH_INFO: `/v3/users` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:02:30.052105 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad1fa56e-6e5c-48ce-8065-d98c2266f5b3 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 22:02:30.052422 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad1fa56e-6e5c-48ce-8065-d98c2266f5b3 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:02:30.053868 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad1fa56e-6e5c-48ce-8065-d98c2266f5b3 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:02:30.073354 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 624/1254] 10.222.0.52 () {56 vars in 1143 bytes} [Wed Jul 13 22:02:29 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 22:02:30.079675 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5852c9-0469-46c6-a81a-3b6c28f26686 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:02:30.192522 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd5852c9-0469-46c6-a81a-3b6c28f26686 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:02:30.232467 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268228f2-1f3e-4323-88de-fd6a24bedb0b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:02:30.233041 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268228f2-1f3e-4323-88de-fd6a24bedb0b None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:02:30.233317 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268228f2-1f3e-4323-88de-fd6a24bedb0b None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:02:30.233588 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-268228f2-1f3e-4323-88de-fd6a24bedb0b None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:02:30.296701 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:02:30.311591 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268228f2-1f3e-4323-88de-fd6a24bedb0b None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:02:30.313170 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-268228f2-1f3e-4323-88de-fd6a24bedb0b None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:02:30.442602 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 631/1255] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Jul 13 22:02:30 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 364 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 13 22:02:30.449759 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1fa56e-6e5c-48ce-8065-d98c2266f5b3 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:02:30.567442 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad1fa56e-6e5c-48ce-8065-d98c2266f5b3 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:02:30.582800 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:02:30.610198 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7350191b-deee-4185-91a0-a90c2b44a83a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:02:30.611041 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7350191b-deee-4185-91a0-a90c2b44a83a None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:02:30.611324 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7350191b-deee-4185-91a0-a90c2b44a83a None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:02:30.611594 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7350191b-deee-4185-91a0-a90c2b44a83a None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:02:30.624995 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7350191b-deee-4185-91a0-a90c2b44a83a None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:02:30.626462 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7350191b-deee-4185-91a0-a90c2b44a83a None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:02:30.645527 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application [None req-7350191b-deee-4185-91a0-a90c2b44a83a None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 22:02:30.645527 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 22:02:30.645527 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 22:02:30.645527 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 22:02:30.645527 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 22:02:30.645527 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 22:02:30.645527 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 22:02:30.645527 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 22:02:30.645527 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 22:02:30.645527 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 22:02:30.645527 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 22:02:30.645527 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 22:02:30.645527 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 22:02:30.645527 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 22:02:30.645527 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 22:02:30.646853 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 22:02:30.646853 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 22:02:30.646853 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 22:02:30.646853 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 22:02:30.646853 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 22:02:30.646853 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 22:02:30.646853 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 22:02:30.646853 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 22:02:30.646853 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application with Lock( Jul 13 22:02:30.646853 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 22:02:30.646853 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._enter() Jul 13 22:02:30.646853 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 22:02:30.646853 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 22:02:30.646853 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 22:02:30.646853 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.creator() Jul 13 22:02:30.646853 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 22:02:30.646853 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application created_value = creator( Jul 13 22:02:30.647968 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 22:02:30.647968 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 22:02:30.647968 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 22:02:30.647968 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Jul 13 22:02:30.647968 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 625/1256] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Jul 13 22:02:30 2022] GET /identity/v3/domains/Default => generated 87 bytes in 198 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 22:02:30.652999 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268228f2-1f3e-4323-88de-fd6a24bedb0b None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:02:30.753958 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:02:30.767014 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-268228f2-1f3e-4323-88de-fd6a24bedb0b None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:02:30.803155 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3d6507-e424-44c2-9492-8f98419ac13b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:02:30.803583 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3d6507-e424-44c2-9492-8f98419ac13b None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:02:30.803897 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3d6507-e424-44c2-9492-8f98419ac13b None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:02:30.804204 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec3d6507-e424-44c2-9492-8f98419ac13b None admin] PATH_INFO: `/v3/domains` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:02:30.804956 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3d6507-e424-44c2-9492-8f98419ac13b None admin] RBAC: Adding query filter params (name=Default) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 22:02:30.805264 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3d6507-e424-44c2-9492-8f98419ac13b None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:02:30.806880 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ec3d6507-e424-44c2-9492-8f98419ac13b None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:02:30.819569 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 632/1257] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Jul 13 22:02:30 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 167 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 22:02:30.825998 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7350191b-deee-4185-91a0-a90c2b44a83a None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:02:30.941048 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7350191b-deee-4185-91a0-a90c2b44a83a None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:02:30.992101 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ccca06-05c7-4dc9-8331-43149c7a325d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:02:30.993163 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ccca06-05c7-4dc9-8331-43149c7a325d None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:02:30.993511 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ccca06-05c7-4dc9-8331-43149c7a325d None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:02:30.993776 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5ccca06-05c7-4dc9-8331-43149c7a325d None admin] PATH_INFO: `/v3/projects/service` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:02:31.001548 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ccca06-05c7-4dc9-8331-43149c7a325d None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:02:31.003189 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5ccca06-05c7-4dc9-8331-43149c7a325d None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:02:31.018851 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application [None req-a5ccca06-05c7-4dc9-8331-43149c7a325d None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 13 22:02:31.018851 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 22:02:31.018851 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 22:02:31.018851 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 22:02:31.018851 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 22:02:31.018851 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 22:02:31.018851 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 22:02:31.018851 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 22:02:31.018851 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 22:02:31.018851 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 22:02:31.018851 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 22:02:31.018851 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 22:02:31.018851 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 13 22:02:31.018851 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 13 22:02:31.018851 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 13 22:02:31.019847 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 13 22:02:31.019847 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 22:02:31.019847 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 22:02:31.019847 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 22:02:31.019847 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 22:02:31.019847 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 22:02:31.019847 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 22:02:31.019847 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 22:02:31.019847 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application with Lock( Jul 13 22:02:31.019847 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 22:02:31.019847 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._enter() Jul 13 22:02:31.019847 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 22:02:31.019847 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 22:02:31.019847 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 22:02:31.019847 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.creator() Jul 13 22:02:31.019847 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 22:02:31.019847 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application created_value = creator( Jul 13 22:02:31.020937 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 13 22:02:31.020937 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 13 22:02:31.020937 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 13 22:02:31.020937 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 13 22:02:31.020937 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 13 22:02:31.020937 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 13 22:02:31.020937 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 13 22:02:31.020937 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Jul 13 22:02:31.020937 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 626/1258] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Jul 13 22:02:30 2022] GET /identity/v3/projects/service => generated 88 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 22:02:31.026784 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3d6507-e424-44c2-9492-8f98419ac13b None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:02:31.142597 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec3d6507-e424-44c2-9492-8f98419ac13b None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:02:31.177475 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdfa37ce-24b9-4265-b85f-5e930b5bddc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:02:31.178203 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfa37ce-24b9-4265-b85f-5e930b5bddc7 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:02:31.178480 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfa37ce-24b9-4265-b85f-5e930b5bddc7 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:02:31.178840 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdfa37ce-24b9-4265-b85f-5e930b5bddc7 None admin] PATH_INFO: `/v3/projects` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:02:31.179816 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdfa37ce-24b9-4265-b85f-5e930b5bddc7 None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 22:02:31.180119 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdfa37ce-24b9-4265-b85f-5e930b5bddc7 None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:02:31.181797 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdfa37ce-24b9-4265-b85f-5e930b5bddc7 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:02:31.195798 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 633/1259] 10.222.0.52 () {56 vars in 1148 bytes} [Wed Jul 13 22:02:31 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 22:02:31.202524 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ccca06-05c7-4dc9-8331-43149c7a325d None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:02:31.231067 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:02:31.317393 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5ccca06-05c7-4dc9-8331-43149c7a325d None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:02:31.359554 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6abbc2c3-0a9b-45af-b9fa-570c70ff9e11 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:02:31.360257 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abbc2c3-0a9b-45af-b9fa-570c70ff9e11 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:02:31.360622 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abbc2c3-0a9b-45af-b9fa-570c70ff9e11 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:02:31.360907 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6abbc2c3-0a9b-45af-b9fa-570c70ff9e11 None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:02:31.361795 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6abbc2c3-0a9b-45af-b9fa-570c70ff9e11 None admin] RBAC: Adding query filter params (role.id=44efba2579f342df8a0789c59d163ec3, scope.project.id=417e3e2f1333481795a089bef2e24bea, user.id=ac673d641d8849c79bf4cf71c9805587) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 22:02:31.362232 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6abbc2c3-0a9b-45af-b9fa-570c70ff9e11 None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:02:31.363908 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6abbc2c3-0a9b-45af-b9fa-570c70ff9e11 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:02:31.371122 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 627/1260] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Jul 13 22:02:31 2022] GET /identity/v3/role_assignments?user.id=ac673d641d8849c79bf4cf71c9805587&scope.project.id=417e3e2f1333481795a089bef2e24bea&role.id=44efba2579f342df8a0789c59d163ec3 => generated 259 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 22:02:33.438424 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b05828-1b07-433b-8395-fe7f4c7ffc8d None None] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:02:33.439008 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b05828-1b07-433b-8395-fe7f4c7ffc8d None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:02:33.439008 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b05828-1b07-433b-8395-fe7f4c7ffc8d None None] PATH_INFO: `/` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:02:33.440396 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 634/1261] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jul 13 22:02:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 22:02:33.445980 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2418b51f-8cf8-4a55-90fb-4a4f46b05b16 None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:02:33.446399 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2418b51f-8cf8-4a55-90fb-4a4f46b05b16 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:02:33.446559 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2418b51f-8cf8-4a55-90fb-4a4f46b05b16 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:02:33.465286 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-2418b51f-8cf8-4a55-90fb-4a4f46b05b16 None None] MFA Rules not processed for user `cc99c93da5074cf5a48bc2aa9a92c5e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 22:02:33.466542 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-2418b51f-8cf8-4a55-90fb-4a4f46b05b16 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:02:33.540549 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 628/1262] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jul 13 22:02:33 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 364 bytes (2 switches on core 0) Jul 13 22:02:33.570395 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b05828-1b07-433b-8395-fe7f4c7ffc8d None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:02:33.582082 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:02:33.691426 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b05828-1b07-433b-8395-fe7f4c7ffc8d None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:02:33.726512 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2bd7916-ffcb-4779-8b19-9375184b3799 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:02:33.727308 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bd7916-ffcb-4779-8b19-9375184b3799 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:02:33.727585 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bd7916-ffcb-4779-8b19-9375184b3799 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:02:33.727858 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2bd7916-ffcb-4779-8b19-9375184b3799 None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:02:33.739763 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2bd7916-ffcb-4779-8b19-9375184b3799 None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 22:02:33.747566 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application [None req-c2bd7916-ffcb-4779-8b19-9375184b3799 None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 13 22:02:33.747566 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 22:02:33.747566 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 22:02:33.747566 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 22:02:33.747566 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 22:02:33.747566 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 22:02:33.747566 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 22:02:33.747566 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 22:02:33.747566 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 22:02:33.747566 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 22:02:33.747566 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 22:02:33.747566 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 22:02:33.747566 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 22:02:33.747566 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 22:02:33.747566 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 22:02:33.747566 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 22:02:33.748622 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 22:02:33.748622 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 22:02:33.748622 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 22:02:33.748622 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 22:02:33.748622 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 22:02:33.748622 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 22:02:33.748622 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 22:02:33.748622 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 22:02:33.748622 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 22:02:33.748622 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 22:02:33.748622 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 22:02:33.748622 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application with Lock( Jul 13 22:02:33.748622 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 22:02:33.748622 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._enter() Jul 13 22:02:33.748622 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 22:02:33.748622 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 22:02:33.750224 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 22:02:33.750224 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.creator() Jul 13 22:02:33.750224 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 22:02:33.750224 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application created_value = creator( Jul 13 22:02:33.750224 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 22:02:33.750224 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 22:02:33.750224 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 22:02:33.750224 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 22:02:33.750224 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 22:02:33.750224 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 22:02:33.750224 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 13 22:02:33.750224 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Jul 13 22:02:33.750224 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 635/1263] 10.222.0.52 () {56 vars in 1096 bytes} [Wed Jul 13 22:02:33 2022] GET /identity/v3/roles/admin => generated 83 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 22:02:33.758668 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2418b51f-8cf8-4a55-90fb-4a4f46b05b16 None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:02:33.780691 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:02:33.865592 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2418b51f-8cf8-4a55-90fb-4a4f46b05b16 None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:02:33.908668 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4809bac-5345-41ca-b3a1-c8cf3755348f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:02:33.909329 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4809bac-5345-41ca-b3a1-c8cf3755348f None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:02:33.909604 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4809bac-5345-41ca-b3a1-c8cf3755348f None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:02:33.909870 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4809bac-5345-41ca-b3a1-c8cf3755348f None admin] PATH_INFO: `/v3/roles` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:02:33.910648 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4809bac-5345-41ca-b3a1-c8cf3755348f None admin] RBAC: Adding query filter params (name=admin) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 22:02:33.910950 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4809bac-5345-41ca-b3a1-c8cf3755348f None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:02:33.912323 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4809bac-5345-41ca-b3a1-c8cf3755348f None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:02:33.920560 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 629/1264] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Jul 13 22:02:33 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 163 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 22:02:33.928976 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2bd7916-ffcb-4779-8b19-9375184b3799 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:02:34.005774 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:02:34.040053 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2bd7916-ffcb-4779-8b19-9375184b3799 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:02:34.074999 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d8ebd7-af3e-4995-a496-27940c63c408 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:02:34.075791 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d8ebd7-af3e-4995-a496-27940c63c408 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:02:34.076069 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d8ebd7-af3e-4995-a496-27940c63c408 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:02:34.076341 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47d8ebd7-af3e-4995-a496-27940c63c408 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:02:34.087461 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d8ebd7-af3e-4995-a496-27940c63c408 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:02:34.088905 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47d8ebd7-af3e-4995-a496-27940c63c408 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:02:34.106108 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application [None req-47d8ebd7-af3e-4995-a496-27940c63c408 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 22:02:34.106108 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 22:02:34.106108 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 22:02:34.106108 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 22:02:34.106108 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 22:02:34.106108 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 22:02:34.106108 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 22:02:34.106108 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 22:02:34.106108 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 22:02:34.106108 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 22:02:34.106108 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 22:02:34.106108 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 22:02:34.106108 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 22:02:34.106108 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 22:02:34.106108 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 22:02:34.107148 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 22:02:34.107148 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 22:02:34.107148 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 22:02:34.107148 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 22:02:34.107148 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 22:02:34.107148 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 22:02:34.107148 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 22:02:34.107148 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 22:02:34.107148 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application with Lock( Jul 13 22:02:34.107148 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 22:02:34.107148 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._enter() Jul 13 22:02:34.107148 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 22:02:34.107148 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 22:02:34.107148 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 22:02:34.107148 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.creator() Jul 13 22:02:34.107148 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 22:02:34.107148 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application created_value = creator( Jul 13 22:02:34.108136 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 22:02:34.108136 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 22:02:34.108136 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 22:02:34.108136 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Jul 13 22:02:34.108136 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 636/1265] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Jul 13 22:02:33 2022] GET /identity/v3/domains/Default => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 22:02:34.113436 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4809bac-5345-41ca-b3a1-c8cf3755348f None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:02:34.195380 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:02:34.216390 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4809bac-5345-41ca-b3a1-c8cf3755348f None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:02:34.250718 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5657bfed-c2d2-4092-800c-d1fdd0c77bb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:02:34.251375 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5657bfed-c2d2-4092-800c-d1fdd0c77bb7 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:02:34.251652 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5657bfed-c2d2-4092-800c-d1fdd0c77bb7 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:02:34.251917 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5657bfed-c2d2-4092-800c-d1fdd0c77bb7 None admin] PATH_INFO: `/v3/domains` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:02:34.252643 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5657bfed-c2d2-4092-800c-d1fdd0c77bb7 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 22:02:34.252943 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5657bfed-c2d2-4092-800c-d1fdd0c77bb7 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:02:34.254307 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5657bfed-c2d2-4092-800c-d1fdd0c77bb7 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:02:34.266300 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 630/1266] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Jul 13 22:02:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 153 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 22:02:34.272432 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d8ebd7-af3e-4995-a496-27940c63c408 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:02:34.384860 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47d8ebd7-af3e-4995-a496-27940c63c408 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:02:34.405725 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:02:34.423794 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66add1eb-872a-4760-8ff9-ff107458cb12 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:02:34.424603 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66add1eb-872a-4760-8ff9-ff107458cb12 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:02:34.424886 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66add1eb-872a-4760-8ff9-ff107458cb12 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:02:34.425164 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66add1eb-872a-4760-8ff9-ff107458cb12 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:02:34.433458 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66add1eb-872a-4760-8ff9-ff107458cb12 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:02:34.434940 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66add1eb-872a-4760-8ff9-ff107458cb12 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:02:34.449623 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application [None req-66add1eb-872a-4760-8ff9-ff107458cb12 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 13 22:02:34.449623 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 22:02:34.449623 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 22:02:34.449623 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 22:02:34.449623 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 22:02:34.449623 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 22:02:34.449623 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 22:02:34.449623 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 22:02:34.449623 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 22:02:34.449623 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 22:02:34.449623 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 22:02:34.449623 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 22:02:34.449623 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 13 22:02:34.449623 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 13 22:02:34.449623 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 13 22:02:34.449623 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 13 22:02:34.451266 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 22:02:34.451266 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 22:02:34.451266 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 13 22:02:34.451266 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 22:02:34.451266 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 13 22:02:34.451266 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 22:02:34.451266 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 22:02:34.451266 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 22:02:34.451266 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 22:02:34.451266 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 22:02:34.451266 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 22:02:34.451266 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application with Lock( Jul 13 22:02:34.451266 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 22:02:34.451266 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._enter() Jul 13 22:02:34.451266 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 22:02:34.451266 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 22:02:34.451266 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 22:02:34.451266 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.creator() Jul 13 22:02:34.452431 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 22:02:34.452431 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application created_value = creator( Jul 13 22:02:34.452431 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 13 22:02:34.452431 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 13 22:02:34.452431 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 13 22:02:34.452431 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 13 22:02:34.452431 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 13 22:02:34.452431 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 13 22:02:34.452431 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 13 22:02:34.452431 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Jul 13 22:02:34.452431 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 637/1267] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Jul 13 22:02:34 2022] GET /identity/v3/users/placement => generated 87 bytes in 179 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 22:02:34.457247 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5657bfed-c2d2-4092-800c-d1fdd0c77bb7 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:02:34.563337 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5657bfed-c2d2-4092-800c-d1fdd0c77bb7 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:02:34.599133 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5b99e8-4814-483b-a987-28c3ced8c113 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:02:34.599798 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5b99e8-4814-483b-a987-28c3ced8c113 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:02:34.600073 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5b99e8-4814-483b-a987-28c3ced8c113 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:02:34.600340 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5b99e8-4814-483b-a987-28c3ced8c113 None admin] PATH_INFO: `/v3/users` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:02:34.601381 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5b99e8-4814-483b-a987-28c3ced8c113 None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 22:02:34.601872 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5b99e8-4814-483b-a987-28c3ced8c113 None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:02:34.603327 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5b99e8-4814-483b-a987-28c3ced8c113 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:02:34.612040 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:02:34.621006 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 631/1268] 10.222.0.52 () {56 vars in 1143 bytes} [Wed Jul 13 22:02:34 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 164 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 22:02:34.627284 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66add1eb-872a-4760-8ff9-ff107458cb12 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:02:34.728795 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66add1eb-872a-4760-8ff9-ff107458cb12 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:02:34.766300 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a147cc26-797b-4a56-a2d6-7e0436a97cc7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:02:34.767116 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a147cc26-797b-4a56-a2d6-7e0436a97cc7 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:02:34.767398 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a147cc26-797b-4a56-a2d6-7e0436a97cc7 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:02:34.767676 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a147cc26-797b-4a56-a2d6-7e0436a97cc7 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:02:34.778684 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a147cc26-797b-4a56-a2d6-7e0436a97cc7 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:02:34.780139 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a147cc26-797b-4a56-a2d6-7e0436a97cc7 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:02:34.797530 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application [None req-a147cc26-797b-4a56-a2d6-7e0436a97cc7 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 22:02:34.797530 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 22:02:34.797530 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 22:02:34.797530 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 22:02:34.797530 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 22:02:34.797530 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 22:02:34.797530 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 22:02:34.797530 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 22:02:34.797530 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 22:02:34.797530 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 22:02:34.797530 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 22:02:34.797530 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 22:02:34.797530 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 22:02:34.797530 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 22:02:34.797530 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 22:02:34.798588 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 22:02:34.798588 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 22:02:34.798588 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 22:02:34.798588 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 22:02:34.798588 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 22:02:34.798588 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 22:02:34.798588 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 22:02:34.798588 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 22:02:34.798588 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application with Lock( Jul 13 22:02:34.798588 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 22:02:34.798588 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._enter() Jul 13 22:02:34.798588 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 22:02:34.798588 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 22:02:34.798588 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 22:02:34.798588 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.creator() Jul 13 22:02:34.798588 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 22:02:34.798588 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application created_value = creator( Jul 13 22:02:34.799658 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 22:02:34.799658 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 22:02:34.799658 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 22:02:34.799658 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Jul 13 22:02:34.799658 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 638/1269] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Jul 13 22:02:34 2022] GET /identity/v3/domains/Default => generated 87 bytes in 172 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 22:02:34.804761 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5b99e8-4814-483b-a987-28c3ced8c113 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:02:34.916983 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5b99e8-4814-483b-a987-28c3ced8c113 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:02:34.959646 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a2611c-28d2-4a62-8d01-997809fdb56a None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:02:34.960279 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a2611c-28d2-4a62-8d01-997809fdb56a None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:02:34.960782 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a2611c-28d2-4a62-8d01-997809fdb56a None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:02:34.961035 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-72a2611c-28d2-4a62-8d01-997809fdb56a None admin] PATH_INFO: `/v3/domains` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:02:34.961789 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a2611c-28d2-4a62-8d01-997809fdb56a None admin] RBAC: Adding query filter params (name=Default) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 22:02:34.962089 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a2611c-28d2-4a62-8d01-997809fdb56a None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:02:34.963488 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-72a2611c-28d2-4a62-8d01-997809fdb56a None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:02:34.974902 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 632/1270] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Jul 13 22:02:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 171 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 22:02:34.981220 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a147cc26-797b-4a56-a2d6-7e0436a97cc7 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:02:35.027610 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:02:35.108617 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a147cc26-797b-4a56-a2d6-7e0436a97cc7 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:02:35.149392 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1849f5a2-4ddf-43f0-8fc0-810497094714 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:02:35.150190 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1849f5a2-4ddf-43f0-8fc0-810497094714 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:02:35.150462 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1849f5a2-4ddf-43f0-8fc0-810497094714 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:02:35.150764 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1849f5a2-4ddf-43f0-8fc0-810497094714 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:02:35.158517 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1849f5a2-4ddf-43f0-8fc0-810497094714 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:02:35.159976 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1849f5a2-4ddf-43f0-8fc0-810497094714 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:02:35.174734 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application [None req-1849f5a2-4ddf-43f0-8fc0-810497094714 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 13 22:02:35.174734 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 22:02:35.174734 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 22:02:35.174734 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 22:02:35.174734 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 22:02:35.174734 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 22:02:35.174734 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 22:02:35.174734 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 22:02:35.174734 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 22:02:35.174734 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 22:02:35.174734 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 22:02:35.174734 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 22:02:35.174734 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 13 22:02:35.174734 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 13 22:02:35.174734 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 13 22:02:35.175822 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 13 22:02:35.175822 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 22:02:35.175822 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 22:02:35.175822 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 22:02:35.175822 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 22:02:35.175822 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 22:02:35.175822 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 22:02:35.175822 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 22:02:35.175822 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application with Lock( Jul 13 22:02:35.175822 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 22:02:35.175822 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._enter() Jul 13 22:02:35.175822 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 22:02:35.175822 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 22:02:35.175822 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 22:02:35.175822 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.creator() Jul 13 22:02:35.175822 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 22:02:35.175822 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application created_value = creator( Jul 13 22:02:35.176945 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 13 22:02:35.176945 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 13 22:02:35.176945 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 13 22:02:35.176945 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 13 22:02:35.176945 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 13 22:02:35.176945 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 13 22:02:35.176945 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 13 22:02:35.176945 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Jul 13 22:02:35.176945 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 639/1271] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Jul 13 22:02:34 2022] GET /identity/v3/projects/service => generated 88 bytes in 195 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 22:02:35.182512 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a2611c-28d2-4a62-8d01-997809fdb56a None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:02:35.245281 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:02:35.294087 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-72a2611c-28d2-4a62-8d01-997809fdb56a None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:02:35.339289 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7acfe4e5-cff0-4ce9-bf4a-cf1665413fdc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:02:35.340157 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acfe4e5-cff0-4ce9-bf4a-cf1665413fdc None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:02:35.340534 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acfe4e5-cff0-4ce9-bf4a-cf1665413fdc None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:02:35.340890 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acfe4e5-cff0-4ce9-bf4a-cf1665413fdc None admin] PATH_INFO: `/v3/projects` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:02:35.341902 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7acfe4e5-cff0-4ce9-bf4a-cf1665413fdc None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 22:02:35.342406 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7acfe4e5-cff0-4ce9-bf4a-cf1665413fdc None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:02:35.344061 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7acfe4e5-cff0-4ce9-bf4a-cf1665413fdc None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:02:35.357709 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 633/1272] 10.222.0.52 () {56 vars in 1148 bytes} [Wed Jul 13 22:02:35 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 22:02:35.365705 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1849f5a2-4ddf-43f0-8fc0-810497094714 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:02:35.463160 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:02:35.473307 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1849f5a2-4ddf-43f0-8fc0-810497094714 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:02:35.510120 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-726f0be9-c730-4c45-9335-a4ddba41c153 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:02:35.511034 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726f0be9-c730-4c45-9335-a4ddba41c153 None admin] REQUEST_METHOD: `PUT` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:02:35.511311 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726f0be9-c730-4c45-9335-a4ddba41c153 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:02:35.511583 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-726f0be9-c730-4c45-9335-a4ddba41c153 None admin] PATH_INFO: `/v3/projects/417e3e2f1333481795a089bef2e24bea/users/ac673d641d8849c79bf4cf71c9805587/roles/44efba2579f342df8a0789c59d163ec3` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:02:35.513764 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-726f0be9-c730-4c45-9335-a4ddba41c153 None admin] RBAC: Authorizing `identity:create_grant(project_id=417e3e2f1333481795a089bef2e24bea, user_id=ac673d641d8849c79bf4cf71c9805587, role_id=44efba2579f342df8a0789c59d163ec3)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:02:35.515246 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-726f0be9-c730-4c45-9335-a4ddba41c153 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:02:35.536040 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 640/1273] 10.222.0.52 () {58 vars in 1439 bytes} [Wed Jul 13 22:02:35 2022] PUT /identity/v3/projects/417e3e2f1333481795a089bef2e24bea/users/ac673d641d8849c79bf4cf71c9805587/roles/44efba2579f342df8a0789c59d163ec3 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 22:02:37.537010 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b534fc-230d-4895-9442-9c2c4029b979 None None] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:02:37.537612 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b534fc-230d-4895-9442-9c2c4029b979 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:02:37.537612 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4b534fc-230d-4895-9442-9c2c4029b979 None None] PATH_INFO: `/` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:02:37.539275 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 634/1274] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jul 13 22:02:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 22:02:37.545501 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c576a76-d66e-49ba-b850-d06f79777a76 None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:02:37.546218 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c576a76-d66e-49ba-b850-d06f79777a76 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:02:37.546341 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0c576a76-d66e-49ba-b850-d06f79777a76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:02:37.569798 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-0c576a76-d66e-49ba-b850-d06f79777a76 None None] MFA Rules not processed for user `cc99c93da5074cf5a48bc2aa9a92c5e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 22:02:37.571083 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-0c576a76-d66e-49ba-b850-d06f79777a76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:02:37.647121 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 641/1275] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jul 13 22:02:37 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 13 22:02:37.677146 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b534fc-230d-4895-9442-9c2c4029b979 None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:02:37.785316 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4b534fc-230d-4895-9442-9c2c4029b979 None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:02:37.803166 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:02:37.821726 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a0a922-b6f0-495e-82fb-a64d94ddc6be None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:02:37.822528 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a0a922-b6f0-495e-82fb-a64d94ddc6be None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:02:37.822822 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a0a922-b6f0-495e-82fb-a64d94ddc6be None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:02:37.823093 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44a0a922-b6f0-495e-82fb-a64d94ddc6be None admin] PATH_INFO: `/v3/roles/admin` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:02:37.834380 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44a0a922-b6f0-495e-82fb-a64d94ddc6be None admin] Extracting inferred target data resulted in "NOT FOUND (404)". {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:409}} Jul 13 22:02:37.842142 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application [None req-44a0a922-b6f0-495e-82fb-a64d94ddc6be None admin] Could not find role: admin.: keystone.exception.RoleNotFound: Could not find role: admin. Jul 13 22:02:37.842142 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 22:02:37.842142 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 22:02:37.842142 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 22:02:37.842142 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 22:02:37.842142 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 22:02:37.842142 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 22:02:37.842142 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 22:02:37.842142 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 22:02:37.842142 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 22:02:37.842142 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 22:02:37.842142 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 22:02:37.842142 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 50, in get Jul 13 22:02:37.842142 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_role(role_id) Jul 13 22:02:37.842142 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/roles.py", line 70, in _get_role Jul 13 22:02:37.842142 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application ENFORCER.enforce_call(action='identity:get_role') Jul 13 22:02:37.843171 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 402, in enforce_call Jul 13 22:02:37.843171 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application policy_dict.update(cls._extract_member_target_data( Jul 13 22:02:37.843171 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py", line 234, in _extract_member_target_data Jul 13 22:02:37.843171 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application member_name: func(flask.request.view_args[key]) Jul 13 22:02:37.843171 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 22:02:37.843171 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 22:02:37.843171 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 22:02:37.843171 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 22:02:37.843171 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 22:02:37.843171 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 22:02:37.843171 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 22:02:37.843171 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application with Lock( Jul 13 22:02:37.843171 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 22:02:37.843171 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._enter() Jul 13 22:02:37.843171 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 22:02:37.843171 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 22:02:37.844275 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 22:02:37.844275 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.creator() Jul 13 22:02:37.844275 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 22:02:37.844275 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application created_value = creator( Jul 13 22:02:37.844275 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/core.py", line 1276, in get_role Jul 13 22:02:37.844275 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.driver.get_role(role_id) Jul 13 22:02:37.844275 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 70, in get_role Jul 13 22:02:37.844275 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_role(session, role_id).to_dict() Jul 13 22:02:37.844275 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/assignment/role_backends/sql.py", line 65, in _get_role Jul 13 22:02:37.844275 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application raise exception.RoleNotFound(role_id=role_id) Jul 13 22:02:37.844275 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application keystone.exception.RoleNotFound: Could not find role: admin. Jul 13 22:02:37.844275 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Jul 13 22:02:37.844275 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 635/1276] 10.222.0.52 () {56 vars in 1096 bytes} [Wed Jul 13 22:02:37 2022] GET /identity/v3/roles/admin => generated 83 bytes in 167 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 22:02:37.849542 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c576a76-d66e-49ba-b850-d06f79777a76 None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:02:37.936044 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:02:37.956496 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0c576a76-d66e-49ba-b850-d06f79777a76 None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:02:37.992793 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c14eae9-a01a-489a-9df2-8d9647ccff49 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:02:37.993450 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c14eae9-a01a-489a-9df2-8d9647ccff49 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:02:37.993725 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c14eae9-a01a-489a-9df2-8d9647ccff49 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:02:37.993997 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c14eae9-a01a-489a-9df2-8d9647ccff49 None admin] PATH_INFO: `/v3/roles` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:02:37.994781 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c14eae9-a01a-489a-9df2-8d9647ccff49 None admin] RBAC: Adding query filter params (name=admin) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 22:02:37.995087 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c14eae9-a01a-489a-9df2-8d9647ccff49 None admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:02:37.996468 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c14eae9-a01a-489a-9df2-8d9647ccff49 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:02:38.004679 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 642/1277] 10.222.0.52 () {56 vars in 1099 bytes} [Wed Jul 13 22:02:37 2022] GET /identity/v3/roles?name=admin => generated 339 bytes in 156 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 22:02:38.010724 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a0a922-b6f0-495e-82fb-a64d94ddc6be None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:02:38.118443 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44a0a922-b6f0-495e-82fb-a64d94ddc6be None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:02:38.153972 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb3c889-9360-4187-bd39-1dd6c8514ffd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:02:38.154776 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb3c889-9360-4187-bd39-1dd6c8514ffd None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:02:38.155050 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb3c889-9360-4187-bd39-1dd6c8514ffd None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:02:38.155314 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbb3c889-9360-4187-bd39-1dd6c8514ffd None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:02:38.166138 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb3c889-9360-4187-bd39-1dd6c8514ffd None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:02:38.167612 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbb3c889-9360-4187-bd39-1dd6c8514ffd None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:02:38.184699 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application [None req-cbb3c889-9360-4187-bd39-1dd6c8514ffd None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 22:02:38.184699 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 22:02:38.184699 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 22:02:38.184699 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 22:02:38.184699 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 22:02:38.184699 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 22:02:38.184699 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 22:02:38.184699 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 22:02:38.184699 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 22:02:38.184699 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 22:02:38.184699 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 22:02:38.184699 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 22:02:38.184699 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 22:02:38.184699 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 22:02:38.184699 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 22:02:38.185735 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 22:02:38.185735 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 22:02:38.185735 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 22:02:38.185735 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 22:02:38.185735 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 22:02:38.185735 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 22:02:38.185735 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 22:02:38.185735 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 22:02:38.185735 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application with Lock( Jul 13 22:02:38.185735 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 22:02:38.185735 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._enter() Jul 13 22:02:38.185735 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 22:02:38.185735 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 22:02:38.185735 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 22:02:38.185735 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.creator() Jul 13 22:02:38.185735 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 22:02:38.185735 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application created_value = creator( Jul 13 22:02:38.186813 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 22:02:38.186813 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 22:02:38.186813 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 22:02:38.186813 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Jul 13 22:02:38.186813 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 636/1278] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Jul 13 22:02:38 2022] GET /identity/v3/domains/Default => generated 87 bytes in 176 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 22:02:38.191911 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c14eae9-a01a-489a-9df2-8d9647ccff49 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:02:38.298572 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c14eae9-a01a-489a-9df2-8d9647ccff49 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:02:38.333915 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bfc7b1f-8866-4009-9626-d899b2fb85d9 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:02:38.334611 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfc7b1f-8866-4009-9626-d899b2fb85d9 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:02:38.334895 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfc7b1f-8866-4009-9626-d899b2fb85d9 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:02:38.335308 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bfc7b1f-8866-4009-9626-d899b2fb85d9 None admin] PATH_INFO: `/v3/domains` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:02:38.336693 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bfc7b1f-8866-4009-9626-d899b2fb85d9 None admin] RBAC: Adding query filter params (name=Default) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 22:02:38.337074 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bfc7b1f-8866-4009-9626-d899b2fb85d9 None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:02:38.338316 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bfc7b1f-8866-4009-9626-d899b2fb85d9 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:02:38.350194 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 643/1279] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Jul 13 22:02:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 159 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 22:02:38.356699 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb3c889-9360-4187-bd39-1dd6c8514ffd None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:02:38.388768 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:02:38.472195 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbb3c889-9360-4187-bd39-1dd6c8514ffd None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:02:38.512268 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b735364-27f2-4ac0-ac16-bf50d863e503 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:02:38.513092 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b735364-27f2-4ac0-ac16-bf50d863e503 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:02:38.513372 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b735364-27f2-4ac0-ac16-bf50d863e503 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:02:38.513638 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b735364-27f2-4ac0-ac16-bf50d863e503 None admin] PATH_INFO: `/v3/users/placement` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:02:38.521260 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b735364-27f2-4ac0-ac16-bf50d863e503 None admin] RBAC: Authorizing `identity:get_user(user_id=placement)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:02:38.522896 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9b735364-27f2-4ac0-ac16-bf50d863e503 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:02:38.537240 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application [None req-9b735364-27f2-4ac0-ac16-bf50d863e503 None admin] Could not find user: placement.: keystone.exception.UserNotFound: Could not find user: placement. Jul 13 22:02:38.537240 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 22:02:38.537240 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 22:02:38.537240 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 22:02:38.537240 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 22:02:38.537240 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 22:02:38.537240 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 22:02:38.537240 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 22:02:38.537240 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 22:02:38.537240 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 22:02:38.537240 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 22:02:38.537240 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 22:02:38.537240 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 182, in get Jul 13 22:02:38.537240 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_user(user_id) Jul 13 22:02:38.537240 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/users.py", line 194, in _get_user Jul 13 22:02:38.537240 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application ref = PROVIDERS.identity_api.get_user(user_id) Jul 13 22:02:38.538826 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 22:02:38.538826 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 22:02:38.538826 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 414, in wrapper Jul 13 22:02:38.538826 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 22:02:38.538826 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 424, in wrapper Jul 13 22:02:38.538826 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return f(self, *args, **kwargs) Jul 13 22:02:38.538826 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 22:02:38.538826 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 22:02:38.538826 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 22:02:38.538826 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 22:02:38.538826 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 22:02:38.538826 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application with Lock( Jul 13 22:02:38.538826 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 22:02:38.538826 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._enter() Jul 13 22:02:38.538826 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 22:02:38.538826 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 22:02:38.538826 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 22:02:38.538826 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.creator() Jul 13 22:02:38.539814 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 22:02:38.539814 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application created_value = creator( Jul 13 22:02:38.539814 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/core.py", line 1011, in get_user Jul 13 22:02:38.539814 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application ref = driver.get_user(entity_id) Jul 13 22:02:38.539814 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 200, in get_user Jul 13 22:02:38.539814 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application self._get_user(session, user_id).to_dict()) Jul 13 22:02:38.539814 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/identity/backends/sql.py", line 194, in _get_user Jul 13 22:02:38.539814 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application raise exception.UserNotFound(user_id=user_id) Jul 13 22:02:38.539814 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application keystone.exception.UserNotFound: Could not find user: placement. Jul 13 22:02:38.539814 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Jul 13 22:02:38.542711 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 637/1280] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Jul 13 22:02:38 2022] GET /identity/v3/users/placement => generated 87 bytes in 186 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 22:02:38.544524 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bfc7b1f-8866-4009-9626-d899b2fb85d9 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:02:38.548612 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:02:38.659039 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bfc7b1f-8866-4009-9626-d899b2fb85d9 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:02:38.703377 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd83c39-356f-4286-b6ec-2e608a1a6ebc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:02:38.704046 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd83c39-356f-4286-b6ec-2e608a1a6ebc None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:02:38.704326 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd83c39-356f-4286-b6ec-2e608a1a6ebc None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:02:38.704608 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd83c39-356f-4286-b6ec-2e608a1a6ebc None admin] PATH_INFO: `/v3/users` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:02:38.705450 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd83c39-356f-4286-b6ec-2e608a1a6ebc None admin] RBAC: Adding query filter params (domain_id=default, name=placement) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 22:02:38.705760 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd83c39-356f-4286-b6ec-2e608a1a6ebc None admin] RBAC: Authorizing `identity:list_users()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:02:38.707249 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd83c39-356f-4286-b6ec-2e608a1a6ebc None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:02:38.724219 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 644/1281] 10.222.0.52 () {56 vars in 1143 bytes} [Wed Jul 13 22:02:38 2022] GET /identity/v3/users?domain_id=default&name=placement => generated 378 bytes in 180 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 22:02:38.733586 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b735364-27f2-4ac0-ac16-bf50d863e503 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:02:38.839499 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:02:38.849597 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9b735364-27f2-4ac0-ac16-bf50d863e503 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:02:38.884221 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413c7af0-9e72-44c1-851c-02f3a7fc95fd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:02:38.884862 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413c7af0-9e72-44c1-851c-02f3a7fc95fd None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:02:38.885137 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413c7af0-9e72-44c1-851c-02f3a7fc95fd None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:02:38.885406 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-413c7af0-9e72-44c1-851c-02f3a7fc95fd None admin] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:02:38.954080 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-413c7af0-9e72-44c1-851c-02f3a7fc95fd None admin] RBAC: Authorizing `identity:validate_token()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:02:38.955479 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-413c7af0-9e72-44c1-851c-02f3a7fc95fd None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:02:39.051313 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:02:39.063876 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 638/1282] 10.222.0.52 () {58 vars in 1282 bytes} [Wed Jul 13 22:02:38 2022] GET /identity/v3/auth/tokens => generated 2108 bytes in 331 msecs (HTTP/1.1 200) 6 headers in 359 bytes (1 switches on core 0) Jul 13 22:02:39.071924 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd83c39-356f-4286-b6ec-2e608a1a6ebc None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:02:39.141343 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:02:39.181067 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd83c39-356f-4286-b6ec-2e608a1a6ebc None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:02:39.214868 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8044ec-84f8-4f6e-aa84-aad94f453bd3 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:02:39.215580 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8044ec-84f8-4f6e-aa84-aad94f453bd3 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:02:39.215869 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8044ec-84f8-4f6e-aa84-aad94f453bd3 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:02:39.216104 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af8044ec-84f8-4f6e-aa84-aad94f453bd3 None admin] PATH_INFO: `/v3/domains/Default` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:02:39.226249 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8044ec-84f8-4f6e-aa84-aad94f453bd3 None admin] RBAC: Authorizing `identity:get_domain(domain_id=Default)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:02:39.227741 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af8044ec-84f8-4f6e-aa84-aad94f453bd3 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:02:39.245163 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application [None req-af8044ec-84f8-4f6e-aa84-aad94f453bd3 None admin] Could not find domain: Default.: keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 22:02:39.245163 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 22:02:39.245163 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 22:02:39.245163 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 22:02:39.245163 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 22:02:39.245163 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 22:02:39.245163 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 22:02:39.245163 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 22:02:39.245163 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 22:02:39.245163 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 22:02:39.245163 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 22:02:39.245163 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 22:02:39.245163 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 89, in get Jul 13 22:02:39.245163 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_domain(domain_id) Jul 13 22:02:39.245163 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/domains.py", line 97, in _get_domain Jul 13 22:02:39.246145 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application domain = PROVIDERS.resource_api.get_domain(domain_id) Jul 13 22:02:39.246145 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 22:02:39.246145 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 22:02:39.246145 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 22:02:39.246145 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 22:02:39.246145 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 22:02:39.246145 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 22:02:39.246145 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 22:02:39.246145 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application with Lock( Jul 13 22:02:39.246145 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 22:02:39.246145 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._enter() Jul 13 22:02:39.246145 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 22:02:39.246145 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 22:02:39.246145 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 22:02:39.246145 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.creator() Jul 13 22:02:39.246145 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 22:02:39.246145 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application created_value = creator( Jul 13 22:02:39.247202 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 716, in get_domain Jul 13 22:02:39.247202 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application raise exception.DomainNotFound(domain_id=domain_id) Jul 13 22:02:39.247202 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application keystone.exception.DomainNotFound: Could not find domain: Default. Jul 13 22:02:39.247202 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Jul 13 22:02:39.247202 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 645/1283] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Jul 13 22:02:39 2022] GET /identity/v3/domains/Default => generated 87 bytes in 175 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 22:02:39.252404 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413c7af0-9e72-44c1-851c-02f3a7fc95fd None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:02:39.361511 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-413c7af0-9e72-44c1-851c-02f3a7fc95fd None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:02:39.396680 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e5131e-6ff6-4da6-8dd7-70a9326bffdd None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:02:39.397337 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e5131e-6ff6-4da6-8dd7-70a9326bffdd None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:02:39.397613 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e5131e-6ff6-4da6-8dd7-70a9326bffdd None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:02:39.397907 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9e5131e-6ff6-4da6-8dd7-70a9326bffdd None admin] PATH_INFO: `/v3/domains` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:02:39.398697 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e5131e-6ff6-4da6-8dd7-70a9326bffdd None admin] RBAC: Adding query filter params (name=Default) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 22:02:39.399003 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e5131e-6ff6-4da6-8dd7-70a9326bffdd None admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:02:39.400387 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9e5131e-6ff6-4da6-8dd7-70a9326bffdd None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:02:39.411666 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 639/1284] 10.222.0.52 () {56 vars in 1109 bytes} [Wed Jul 13 22:02:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 160 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 22:02:39.420429 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8044ec-84f8-4f6e-aa84-aad94f453bd3 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:02:39.523625 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af8044ec-84f8-4f6e-aa84-aad94f453bd3 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:02:39.544967 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:02:39.567984 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968b319e-afb1-4d2e-a15b-582c49545fb7 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:02:39.568776 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968b319e-afb1-4d2e-a15b-582c49545fb7 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:02:39.569051 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968b319e-afb1-4d2e-a15b-582c49545fb7 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:02:39.569324 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-968b319e-afb1-4d2e-a15b-582c49545fb7 None admin] PATH_INFO: `/v3/projects/service` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:02:39.576433 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968b319e-afb1-4d2e-a15b-582c49545fb7 None admin] RBAC: Authorizing `identity:get_project(project_id=service)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:02:39.577872 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-968b319e-afb1-4d2e-a15b-582c49545fb7 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:02:39.591373 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application [None req-968b319e-afb1-4d2e-a15b-582c49545fb7 None admin] Could not find project: service.: keystone.exception.ProjectNotFound: Could not find project: service. Jul 13 22:02:39.591373 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 22:02:39.591373 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 22:02:39.591373 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 22:02:39.591373 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 22:02:39.591373 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 22:02:39.591373 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 22:02:39.591373 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 22:02:39.591373 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 22:02:39.591373 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 22:02:39.591373 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 22:02:39.591373 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 22:02:39.591373 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 151, in get Jul 13 22:02:39.591373 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_project(project_id) Jul 13 22:02:39.591373 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/projects.py", line 107, in _get_project Jul 13 22:02:39.592413 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application project = PROVIDERS.resource_api.get_project(project_id) Jul 13 22:02:39.592413 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 22:02:39.592413 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 22:02:39.592413 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 22:02:39.592413 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 22:02:39.592413 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 22:02:39.592413 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 22:02:39.592413 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 22:02:39.592413 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application with Lock( Jul 13 22:02:39.592413 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 22:02:39.592413 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._enter() Jul 13 22:02:39.592413 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 22:02:39.592413 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 22:02:39.592413 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 22:02:39.592413 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.creator() Jul 13 22:02:39.592413 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 22:02:39.592413 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application created_value = creator( Jul 13 22:02:39.593353 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/core.py", line 903, in get_project Jul 13 22:02:39.593353 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self.driver.get_project(project_id) Jul 13 22:02:39.593353 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 49, in get_project Jul 13 22:02:39.593353 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application return self._get_project(session, project_id).to_dict() Jul 13 22:02:39.593353 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/resource/backends/sql.py", line 44, in _get_project Jul 13 22:02:39.593353 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application raise exception.ProjectNotFound(project_id=project_id) Jul 13 22:02:39.593353 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application keystone.exception.ProjectNotFound: Could not find project: service. Jul 13 22:02:39.593353 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: ERROR keystone.server.flask.application Jul 13 22:02:39.593353 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 646/1285] 10.222.0.52 () {56 vars in 1111 bytes} [Wed Jul 13 22:02:39 2022] GET /identity/v3/projects/service => generated 88 bytes in 173 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 22:02:39.600406 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e5131e-6ff6-4da6-8dd7-70a9326bffdd None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:02:39.655930 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:02:39.717315 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9e5131e-6ff6-4da6-8dd7-70a9326bffdd None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:02:39.758390 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7d789fe-0d8a-4a6d-b642-d03fd5f28e4f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:02:39.759068 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d789fe-0d8a-4a6d-b642-d03fd5f28e4f None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:02:39.759347 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d789fe-0d8a-4a6d-b642-d03fd5f28e4f None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:02:39.759620 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7d789fe-0d8a-4a6d-b642-d03fd5f28e4f None admin] PATH_INFO: `/v3/projects` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:02:39.760408 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d789fe-0d8a-4a6d-b642-d03fd5f28e4f None admin] RBAC: Adding query filter params (domain_id=default, name=service) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 22:02:39.760716 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d789fe-0d8a-4a6d-b642-d03fd5f28e4f None admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:02:39.762157 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7d789fe-0d8a-4a6d-b642-d03fd5f28e4f None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:02:39.774796 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 640/1286] 10.222.0.52 () {56 vars in 1148 bytes} [Wed Jul 13 22:02:39 2022] GET /identity/v3/projects?domain_id=default&name=service => generated 429 bytes in 175 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 22:02:39.781194 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968b319e-afb1-4d2e-a15b-582c49545fb7 None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:02:39.884905 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-968b319e-afb1-4d2e-a15b-582c49545fb7 None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:02:39.922275 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3d2a4e0-f5a9-4b75-a917-033b591823cf None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:02:39.922961 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d2a4e0-f5a9-4b75-a917-033b591823cf None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:02:39.923244 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d2a4e0-f5a9-4b75-a917-033b591823cf None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:02:39.923518 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3d2a4e0-f5a9-4b75-a917-033b591823cf None admin] PATH_INFO: `/v3/role_assignments` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:02:39.924349 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d2a4e0-f5a9-4b75-a917-033b591823cf None admin] RBAC: Adding query filter params (role.id=44efba2579f342df8a0789c59d163ec3, scope.project.id=417e3e2f1333481795a089bef2e24bea, user.id=ac673d641d8849c79bf4cf71c9805587) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 22:02:39.924660 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d2a4e0-f5a9-4b75-a917-033b591823cf None admin] RBAC: Authorizing `identity:list_role_assignments()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:02:39.926100 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3d2a4e0-f5a9-4b75-a917-033b591823cf None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:02:39.932505 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 647/1287] 10.222.0.52 () {56 vars in 1374 bytes} [Wed Jul 13 22:02:39 2022] GET /identity/v3/role_assignments?user.id=ac673d641d8849c79bf4cf71c9805587&scope.project.id=417e3e2f1333481795a089bef2e24bea&role.id=44efba2579f342df8a0789c59d163ec3 => generated 608 bytes in 152 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 22:02:41.937937 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ab656e-a590-45ba-8bf7-cd4ed890e253 None None] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:02:41.938607 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ab656e-a590-45ba-8bf7-cd4ed890e253 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:02:41.938607 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28ab656e-a590-45ba-8bf7-cd4ed890e253 None None] PATH_INFO: `/` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:02:41.940551 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 641/1288] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jul 13 22:02:41 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 22:02:41.948954 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db4b404-4327-4266-ab5e-f212a86120ef None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:02:41.949408 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db4b404-4327-4266-ab5e-f212a86120ef None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:02:41.949507 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5db4b404-4327-4266-ab5e-f212a86120ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:02:41.968705 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-5db4b404-4327-4266-ab5e-f212a86120ef None None] MFA Rules not processed for user `cc99c93da5074cf5a48bc2aa9a92c5e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 22:02:41.969886 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-5db4b404-4327-4266-ab5e-f212a86120ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:02:42.001588 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:02:42.045555 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 648/1289] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jul 13 22:02:41 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 13 22:02:42.077386 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ab656e-a590-45ba-8bf7-cd4ed890e253 None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:02:42.187017 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28ab656e-a590-45ba-8bf7-cd4ed890e253 None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:02:42.199033 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:02:42.236665 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2bd4bc-1bb8-4ece-83e5-a94fa182a827 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:02:42.237462 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2bd4bc-1bb8-4ece-83e5-a94fa182a827 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:02:42.237738 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2bd4bc-1bb8-4ece-83e5-a94fa182a827 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:02:42.238009 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f2bd4bc-1bb8-4ece-83e5-a94fa182a827 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:02:42.238700 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f2bd4bc-1bb8-4ece-83e5-a94fa182a827 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:02:42.240047 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f2bd4bc-1bb8-4ece-83e5-a94fa182a827 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:02:42.252229 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application [None req-8f2bd4bc-1bb8-4ece-83e5-a94fa182a827 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 13 22:02:42.252229 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 22:02:42.252229 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 13 22:02:42.252229 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 13 22:02:42.252229 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 13 22:02:42.252229 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 13 22:02:42.252229 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 13 22:02:42.252229 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 13 22:02:42.252229 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 13 22:02:42.252229 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Jul 13 22:02:42.252229 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 13 22:02:42.252229 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Jul 13 22:02:42.252229 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 22:02:42.252229 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 22:02:42.252229 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 22:02:42.252229 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 22:02:42.252229 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 22:02:42.253389 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 22:02:42.253389 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 22:02:42.253389 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 22:02:42.253389 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 22:02:42.253389 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 22:02:42.253389 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 22:02:42.253389 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 13 22:02:42.253389 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 13 22:02:42.253389 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 13 22:02:42.253389 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 13 22:02:42.253389 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 22:02:42.253389 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 22:02:42.253389 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 22:02:42.253389 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 22:02:42.253389 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 22:02:42.253389 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 22:02:42.254423 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 22:02:42.254423 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application with Lock( Jul 13 22:02:42.254423 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 22:02:42.254423 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._enter() Jul 13 22:02:42.254423 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 22:02:42.254423 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 22:02:42.254423 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 22:02:42.254423 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.creator() Jul 13 22:02:42.254423 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 22:02:42.254423 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application created_value = creator( Jul 13 22:02:42.254423 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 13 22:02:42.254423 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 13 22:02:42.254423 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 13 22:02:42.254423 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Jul 13 22:02:42.254423 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 642/1290] 10.222.0.52 () {56 vars in 1117 bytes} [Wed Jul 13 22:02:42 2022] GET /identity/v3/services/placement => generated 90 bytes in 177 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 22:02:42.260232 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db4b404-4327-4266-ab5e-f212a86120ef None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:02:42.369510 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5db4b404-4327-4266-ab5e-f212a86120ef None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:02:42.411635 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d85db6-6c38-40e6-82bd-fdd27e173ccc None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:02:42.412335 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d85db6-6c38-40e6-82bd-fdd27e173ccc None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:02:42.412615 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d85db6-6c38-40e6-82bd-fdd27e173ccc None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:02:42.412885 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d85db6-6c38-40e6-82bd-fdd27e173ccc None admin] PATH_INFO: `/v3/services` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:02:42.413776 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d85db6-6c38-40e6-82bd-fdd27e173ccc None admin] RBAC: Adding query filter params (name=placement) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 22:02:42.414233 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d85db6-6c38-40e6-82bd-fdd27e173ccc None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:02:42.415651 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d85db6-6c38-40e6-82bd-fdd27e173ccc None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:02:42.420633 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 649/1291] 10.222.0.52 () {56 vars in 1116 bytes} [Wed Jul 13 22:02:42 2022] GET /identity/v3/services?name=placement => generated 126 bytes in 161 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 22:02:42.426705 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2bd4bc-1bb8-4ece-83e5-a94fa182a827 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:02:42.529176 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f2bd4bc-1bb8-4ece-83e5-a94fa182a827 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:02:42.564303 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ca732a9-a92b-46ad-898a-5115747d9a55 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:02:42.564964 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca732a9-a92b-46ad-898a-5115747d9a55 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:02:42.565247 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca732a9-a92b-46ad-898a-5115747d9a55 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:02:42.565515 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca732a9-a92b-46ad-898a-5115747d9a55 None admin] PATH_INFO: `/v3/services` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:02:42.566230 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca732a9-a92b-46ad-898a-5115747d9a55 None admin] RBAC: Adding query filter params (type=placement) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 22:02:42.566536 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca732a9-a92b-46ad-898a-5115747d9a55 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:02:42.567937 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ca732a9-a92b-46ad-898a-5115747d9a55 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:02:42.572977 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 643/1292] 10.222.0.52 () {56 vars in 1116 bytes} [Wed Jul 13 22:02:42 2022] GET /identity/v3/services?type=placement => generated 126 bytes in 147 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 22:02:44.566317 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b0e626-6831-4f5a-bb0d-860abb779acd None None] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:02:44.566821 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b0e626-6831-4f5a-bb0d-860abb779acd None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:02:44.566941 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88b0e626-6831-4f5a-bb0d-860abb779acd None None] PATH_INFO: `/` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:02:44.568261 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 650/1293] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jul 13 22:02:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 22:02:44.574781 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c345532-c264-41bd-a500-8fd59f2ac516 None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:02:44.575226 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c345532-c264-41bd-a500-8fd59f2ac516 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:02:44.575327 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c345532-c264-41bd-a500-8fd59f2ac516 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:02:44.595864 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-7c345532-c264-41bd-a500-8fd59f2ac516 None None] MFA Rules not processed for user `cc99c93da5074cf5a48bc2aa9a92c5e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 22:02:44.597058 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-7c345532-c264-41bd-a500-8fd59f2ac516 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:02:44.629648 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:02:44.669793 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 644/1294] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jul 13 22:02:44 2022] POST /identity/v3/auth/tokens => generated 2108 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 13 22:02:44.692842 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b0e626-6831-4f5a-bb0d-860abb779acd None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:02:44.728273 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:02:44.813602 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88b0e626-6831-4f5a-bb0d-860abb779acd None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:02:44.848573 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-883fa703-0df0-42e6-9eb1-09b8ed805b9b None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:02:44.849232 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883fa703-0df0-42e6-9eb1-09b8ed805b9b None admin] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:02:44.849516 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883fa703-0df0-42e6-9eb1-09b8ed805b9b None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:02:44.849772 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-883fa703-0df0-42e6-9eb1-09b8ed805b9b None admin] PATH_INFO: `/v3/services` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:02:44.850527 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883fa703-0df0-42e6-9eb1-09b8ed805b9b None admin] RBAC: Authorizing `identity:create_service()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:02:44.852024 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-883fa703-0df0-42e6-9eb1-09b8ed805b9b None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:02:44.868977 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 651/1295] 10.222.0.52 () {60 vars in 1141 bytes} [Wed Jul 13 22:02:44 2022] POST /identity/v3/services => generated 246 bytes in 177 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 22:02:46.935468 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb24680-68fc-4ba9-bf1e-86995ae436dd None None] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:02:46.936359 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb24680-68fc-4ba9-bf1e-86995ae436dd None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:02:46.936359 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bb24680-68fc-4ba9-bf1e-86995ae436dd None None] PATH_INFO: `/` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:02:46.938547 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 645/1296] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jul 13 22:02:46 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 22:02:46.947122 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9288abc0-309c-4c83-99dc-592ec7bd8330 None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:02:46.947845 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9288abc0-309c-4c83-99dc-592ec7bd8330 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:02:46.948022 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9288abc0-309c-4c83-99dc-592ec7bd8330 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:02:46.970303 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-9288abc0-309c-4c83-99dc-592ec7bd8330 None None] MFA Rules not processed for user `cc99c93da5074cf5a48bc2aa9a92c5e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 22:02:46.971505 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-9288abc0-309c-4c83-99dc-592ec7bd8330 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:02:47.069198 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 652/1297] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jul 13 22:02:46 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 13 22:02:47.098656 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb24680-68fc-4ba9-bf1e-86995ae436dd None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:02:47.207953 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bb24680-68fc-4ba9-bf1e-86995ae436dd None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:02:47.242704 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed04bbd-5da4-4715-ae6b-c3d153337446 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:02:47.243496 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed04bbd-5da4-4715-ae6b-c3d153337446 None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:02:47.243765 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed04bbd-5da4-4715-ae6b-c3d153337446 None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:02:47.244032 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bed04bbd-5da4-4715-ae6b-c3d153337446 None admin] PATH_INFO: `/v3/services/placement` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:02:47.244684 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed04bbd-5da4-4715-ae6b-c3d153337446 None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:02:47.246051 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bed04bbd-5da4-4715-ae6b-c3d153337446 None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:02:47.258265 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application [None req-bed04bbd-5da4-4715-ae6b-c3d153337446 None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 13 22:02:47.258265 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 22:02:47.258265 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 13 22:02:47.258265 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 13 22:02:47.258265 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 13 22:02:47.258265 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 13 22:02:47.258265 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 13 22:02:47.258265 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 13 22:02:47.258265 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 13 22:02:47.258265 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Jul 13 22:02:47.258265 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 13 22:02:47.258265 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Jul 13 22:02:47.258265 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 22:02:47.258265 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 22:02:47.258265 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 22:02:47.258265 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 22:02:47.258265 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 22:02:47.259435 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 22:02:47.259435 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 22:02:47.259435 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 22:02:47.259435 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 22:02:47.259435 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 22:02:47.259435 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 22:02:47.259435 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 13 22:02:47.259435 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 13 22:02:47.259435 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 13 22:02:47.259435 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 13 22:02:47.259435 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 22:02:47.259435 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 22:02:47.259435 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 22:02:47.259435 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 22:02:47.259435 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 22:02:47.259435 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 22:02:47.260282 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 22:02:47.260282 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application with Lock( Jul 13 22:02:47.260282 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 22:02:47.260282 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._enter() Jul 13 22:02:47.260282 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 22:02:47.260282 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 22:02:47.260282 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 22:02:47.260282 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.creator() Jul 13 22:02:47.260282 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 22:02:47.260282 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application created_value = creator( Jul 13 22:02:47.260282 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 13 22:02:47.260282 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 13 22:02:47.260282 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 13 22:02:47.260282 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Jul 13 22:02:47.260282 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 646/1298] 10.222.0.52 () {56 vars in 1117 bytes} [Wed Jul 13 22:02:47 2022] GET /identity/v3/services/placement => generated 90 bytes in 161 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 22:02:47.268188 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9288abc0-309c-4c83-99dc-592ec7bd8330 None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:02:47.294276 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:02:47.393079 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9288abc0-309c-4c83-99dc-592ec7bd8330 None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:02:47.428175 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4c0f09-6d5d-4039-b6dc-69a284ab456f None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:02:47.428960 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4c0f09-6d5d-4039-b6dc-69a284ab456f None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:02:47.429287 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4c0f09-6d5d-4039-b6dc-69a284ab456f None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:02:47.429620 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed4c0f09-6d5d-4039-b6dc-69a284ab456f None admin] PATH_INFO: `/v3/services` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:02:47.430335 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4c0f09-6d5d-4039-b6dc-69a284ab456f None admin] RBAC: Adding query filter params (name=placement) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 22:02:47.430669 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4c0f09-6d5d-4039-b6dc-69a284ab456f None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:02:47.432044 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed4c0f09-6d5d-4039-b6dc-69a284ab456f None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:02:47.437291 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 653/1299] 10.222.0.52 () {56 vars in 1116 bytes} [Wed Jul 13 22:02:47 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 170 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 22:02:47.443633 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed04bbd-5da4-4715-ae6b-c3d153337446 None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:02:47.480089 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:02:47.552450 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bed04bbd-5da4-4715-ae6b-c3d153337446 None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:02:47.591382 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd16884f-e78c-4f53-bd74-a71cb667131e None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:02:47.592028 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd16884f-e78c-4f53-bd74-a71cb667131e None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:02:47.592301 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd16884f-e78c-4f53-bd74-a71cb667131e None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:02:47.592569 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd16884f-e78c-4f53-bd74-a71cb667131e None admin] PATH_INFO: `/v3/endpoints` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:02:47.593353 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd16884f-e78c-4f53-bd74-a71cb667131e None admin] RBAC: Adding query filter params (interface=public, service_id=b94e937c60ce4fba92f05c816d02ee76, region_id=RegionOne) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 22:02:47.593656 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd16884f-e78c-4f53-bd74-a71cb667131e None admin] RBAC: Authorizing `identity:list_endpoints()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:02:47.595054 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd16884f-e78c-4f53-bd74-a71cb667131e None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:02:47.600162 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 647/1300] 10.222.0.52 () {56 vars in 1251 bytes} [Wed Jul 13 22:02:47 2022] GET /identity/v3/endpoints?service_id=b94e937c60ce4fba92f05c816d02ee76&interface=public®ion_id=RegionOne => generated 194 bytes in 157 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 22:02:47.606804 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4c0f09-6d5d-4039-b6dc-69a284ab456f None admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:02:47.696661 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:02:47.716885 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed4c0f09-6d5d-4039-b6dc-69a284ab456f None admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:02:47.759860 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c74730e-4dd6-4caa-9626-c04ecac6e1b8 None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:02:47.760811 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c74730e-4dd6-4caa-9626-c04ecac6e1b8 None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:02:47.761261 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c74730e-4dd6-4caa-9626-c04ecac6e1b8 None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:02:47.761700 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c74730e-4dd6-4caa-9626-c04ecac6e1b8 None admin] PATH_INFO: `/v3/services` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:02:47.762847 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c74730e-4dd6-4caa-9626-c04ecac6e1b8 None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:02:47.765057 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c74730e-4dd6-4caa-9626-c04ecac6e1b8 None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:02:47.775111 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 654/1301] 10.222.0.52 () {56 vars in 1087 bytes} [Wed Jul 13 22:02:47 2022] GET /identity/v3/services => generated 2191 bytes in 169 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 22:02:49.782584 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae05eb35-7d74-4661-9723-9f7dd967ce84 None None] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:02:49.783179 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae05eb35-7d74-4661-9723-9f7dd967ce84 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:02:49.783179 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae05eb35-7d74-4661-9723-9f7dd967ce84 None None] PATH_INFO: `/` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:02:49.784531 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 648/1302] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jul 13 22:02:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 22:02:49.791419 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b90723-7ba3-46be-b59f-cbb71731078a None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:02:49.792098 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b90723-7ba3-46be-b59f-cbb71731078a None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:02:49.792308 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7b90723-7ba3-46be-b59f-cbb71731078a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:02:49.820238 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-d7b90723-7ba3-46be-b59f-cbb71731078a None None] MFA Rules not processed for user `cc99c93da5074cf5a48bc2aa9a92c5e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 22:02:49.821935 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-d7b90723-7ba3-46be-b59f-cbb71731078a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:02:49.904074 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 655/1303] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jul 13 22:02:49 2022] POST /identity/v3/auth/tokens => generated 2211 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 364 bytes (1 switches on core 0) Jul 13 22:02:49.927284 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae05eb35-7d74-4661-9723-9f7dd967ce84 None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:02:50.032497 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:02:50.045525 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae05eb35-7d74-4661-9723-9f7dd967ce84 None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:02:50.084492 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f199889-7523-48c9-b128-7611b0d578df None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:02:50.085293 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f199889-7523-48c9-b128-7611b0d578df None admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:02:50.085743 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f199889-7523-48c9-b128-7611b0d578df None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:02:50.086017 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f199889-7523-48c9-b128-7611b0d578df None admin] PATH_INFO: `/v3/services/placement` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:02:50.086711 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f199889-7523-48c9-b128-7611b0d578df None admin] RBAC: Authorizing `identity:get_service(service_id=placement)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:02:50.088085 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6f199889-7523-48c9-b128-7611b0d578df None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:02:50.099891 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application [None req-6f199889-7523-48c9-b128-7611b0d578df None admin] Could not find service: placement.: keystone.exception.ServiceNotFound: Could not find service: placement. Jul 13 22:02:50.099891 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 22:02:50.099891 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 152, in get_service Jul 13 22:02:50.099891 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.driver.get_service(service_id) Jul 13 22:02:50.099891 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 196, in get_service Jul 13 22:02:50.099891 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_service(session, service_id).to_dict() Jul 13 22:02:50.099891 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/backends/sql.py", line 191, in _get_service Jul 13 22:02:50.099891 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 13 22:02:50.099891 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 13 22:02:50.099891 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Jul 13 22:02:50.099891 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application During handling of the above exception, another exception occurred: Jul 13 22:02:50.099891 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Jul 13 22:02:50.099891 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Traceback (most recent call last): Jul 13 22:02:50.099891 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1523, in full_dispatch_request Jul 13 22:02:50.099891 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application rv = self.dispatch_request() Jul 13 22:02:50.099891 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/app.py", line 1509, in dispatch_request Jul 13 22:02:50.099891 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.ensure_sync(self.view_functions[rule.endpoint])(**req.view_args) Jul 13 22:02:50.101038 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 467, in wrapper Jul 13 22:02:50.101038 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = resource(*args, **kwargs) Jul 13 22:02:50.101038 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask/views.py", line 84, in view Jul 13 22:02:50.101038 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return current_app.ensure_sync(self.dispatch_request)(*args, **kwargs) Jul 13 22:02:50.101038 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/flask_restful/__init__.py", line 582, in dispatch_request Jul 13 22:02:50.101038 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application resp = meth(*args, **kwargs) Jul 13 22:02:50.101038 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 45, in get Jul 13 22:02:50.101038 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._get_service(service_id) Jul 13 22:02:50.101038 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/api/services.py", line 34, in _get_service Jul 13 22:02:50.101038 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.wrap_member(PROVIDERS.catalog_api.get_service(service_id)) Jul 13 22:02:50.101038 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/common/manager.py", line 115, in wrapped Jul 13 22:02:50.101038 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application __ret_val = __f(*args, **kwargs) Jul 13 22:02:50.101038 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/decorator.py", line 232, in fun Jul 13 22:02:50.101038 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return caller(func, *(extras + args), **kw) Jul 13 22:02:50.101038 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1577, in get_or_create_for_user_func Jul 13 22:02:50.101038 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.get_or_create( Jul 13 22:02:50.101976 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 1042, in get_or_create Jul 13 22:02:50.101976 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application with Lock( Jul 13 22:02:50.101976 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 185, in __enter__ Jul 13 22:02:50.101976 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self._enter() Jul 13 22:02:50.101976 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 94, in _enter Jul 13 22:02:50.101976 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application generated = self._enter_create(value, createdtime) Jul 13 22:02:50.101976 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/lock.py", line 178, in _enter_create Jul 13 22:02:50.101976 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application return self.creator() Jul 13 22:02:50.101976 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/usr/local/lib/python3.8/dist-packages/dogpile/cache/region.py", line 995, in gen_value Jul 13 22:02:50.101976 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application created_value = creator( Jul 13 22:02:50.101976 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application File "/opt/stack/keystone/keystone/catalog/core.py", line 154, in get_service Jul 13 22:02:50.101976 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application raise exception.ServiceNotFound(service_id=service_id) Jul 13 22:02:50.101976 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application keystone.exception.ServiceNotFound: Could not find service: placement. Jul 13 22:02:50.101976 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: ERROR keystone.server.flask.application Jul 13 22:02:50.101976 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 649/1304] 10.222.0.52 () {56 vars in 1117 bytes} [Wed Jul 13 22:02:49 2022] GET /identity/v3/services/placement => generated 90 bytes in 174 msecs (HTTP/1.1 404) 5 headers in 183 bytes (1 switches on core 0) Jul 13 22:02:50.108051 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b90723-7ba3-46be-b59f-cbb71731078a None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:02:50.189777 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:02:50.214106 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7b90723-7ba3-46be-b59f-cbb71731078a None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:02:50.249078 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4f2cf53-e874-484d-af3c-acd28b87177d None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:02:50.249730 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f2cf53-e874-484d-af3c-acd28b87177d None admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:02:50.250005 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f2cf53-e874-484d-af3c-acd28b87177d None admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:02:50.250277 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4f2cf53-e874-484d-af3c-acd28b87177d None admin] PATH_INFO: `/v3/services` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:02:50.251016 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f2cf53-e874-484d-af3c-acd28b87177d None admin] RBAC: Adding query filter params (name=placement) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 22:02:50.251315 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f2cf53-e874-484d-af3c-acd28b87177d None admin] RBAC: Authorizing `identity:list_services()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:02:50.252682 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4f2cf53-e874-484d-af3c-acd28b87177d None admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:02:50.257870 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 656/1305] 10.222.0.52 () {56 vars in 1116 bytes} [Wed Jul 13 22:02:50 2022] GET /identity/v3/services?name=placement => generated 358 bytes in 151 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 22:02:50.264298 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f199889-7523-48c9-b128-7611b0d578df None admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:02:50.386722 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6f199889-7523-48c9-b128-7611b0d578df None admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:02:50.421443 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bff6c055-e4a5-40eb-a8d4-3f4c47589bee None admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': 'all', 'project_id': None, 'project_domain_id': None, 'roles': ['admin', 'member', 'reader'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:02:50.422093 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff6c055-e4a5-40eb-a8d4-3f4c47589bee None admin] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:02:50.422371 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff6c055-e4a5-40eb-a8d4-3f4c47589bee None admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:02:50.422675 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bff6c055-e4a5-40eb-a8d4-3f4c47589bee None admin] PATH_INFO: `/v3/endpoints` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:02:50.423420 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff6c055-e4a5-40eb-a8d4-3f4c47589bee None admin] RBAC: Authorizing `identity:create_endpoint()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:02:50.424795 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bff6c055-e4a5-40eb-a8d4-3f4c47589bee None admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:02:50.446948 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 650/1306] 10.222.0.52 () {60 vars in 1144 bytes} [Wed Jul 13 22:02:50 2022] POST /identity/v3/endpoints => generated 331 bytes in 184 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 22:03:00.255220 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=30 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:03:00.441784 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=91,keystone:INSERT=1 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:03:54.138447 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4085a51-c3d1-4d7e-8ba4-d8adcd83b8ab None None] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:03:54.139702 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4085a51-c3d1-4d7e-8ba4-d8adcd83b8ab None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:03:54.140387 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4085a51-c3d1-4d7e-8ba4-d8adcd83b8ab None None] PATH_INFO: `/` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:03:54.149475 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8530d175-b0ff-41e6-a05d-bfa8a11511d2 None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:03:54.149918 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8530d175-b0ff-41e6-a05d-bfa8a11511d2 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:03:54.150194 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8530d175-b0ff-41e6-a05d-bfa8a11511d2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:03:54.152565 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 657/1307] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jul 13 22:03:54 2022] GET /identity => generated 270 bytes in 15 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 22:03:54.272241 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-8530d175-b0ff-41e6-a05d-bfa8a11511d2 None None] MFA Rules not processed for user `cc99c93da5074cf5a48bc2aa9a92c5e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 22:03:54.273957 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-8530d175-b0ff-41e6-a05d-bfa8a11511d2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:03:54.416319 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 651/1308] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jul 13 22:03:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 267 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 22:03:54.457788 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a626e048-837c-4cfb-a690-53a866469b97 None None] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:03:54.458068 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a626e048-837c-4cfb-a690-53a866469b97 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:03:54.458348 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a626e048-837c-4cfb-a690-53a866469b97 None None] PATH_INFO: `/` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:03:54.459936 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 658/1309] 10.222.0.52 () {54 vars in 964 bytes} [Wed Jul 13 22:03:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 22:03:54.466305 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73db4b7d-58b5-4b65-a378-1214c29bf0dc None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:03:54.466625 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73db4b7d-58b5-4b65-a378-1214c29bf0dc None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:03:54.466902 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73db4b7d-58b5-4b65-a378-1214c29bf0dc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:03:54.544142 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-73db4b7d-58b5-4b65-a378-1214c29bf0dc None None] MFA Rules not processed for user `847ec9a1bd4d4dec804318a5c845895a`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 22:03:54.545509 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-73db4b7d-58b5-4b65-a378-1214c29bf0dc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:03:54.571554 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:03:54.631065 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 652/1310] 10.222.0.52 () {58 vars in 1061 bytes} [Wed Jul 13 22:03:54 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 22:03:54.639811 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a626e048-837c-4cfb-a690-53a866469b97 None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:03:54.641559 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-a626e048-837c-4cfb-a690-53a866469b97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:03:54.752177 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a626e048-837c-4cfb-a690-53a866469b97 None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:03:54.828431 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6449bd7-472a-4c57-983b-66e93f828e97 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '847ec9a1bd4d4dec804318a5c845895a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417e3e2f1333481795a089bef2e24bea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:03:54.829090 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6449bd7-472a-4c57-983b-66e93f828e97 service neutron] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:03:54.829354 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6449bd7-472a-4c57-983b-66e93f828e97 service neutron] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:03:54.829627 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6449bd7-472a-4c57-983b-66e93f828e97 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:03:54.831306 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-c6449bd7-472a-4c57-983b-66e93f828e97 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:03:54.887810 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:03:54.944617 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6449bd7-472a-4c57-983b-66e93f828e97 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:03:54.946070 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6449bd7-472a-4c57-983b-66e93f828e97 service neutron] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:03:54.965736 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 659/1311] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jul 13 22:03:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 22:03:57.283714 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c6c720-a19c-42f7-ba8f-cedf2f80d3ae None None] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:03:57.284298 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c6c720-a19c-42f7-ba8f-cedf2f80d3ae None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:03:57.284298 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5c6c720-a19c-42f7-ba8f-cedf2f80d3ae None None] PATH_INFO: `/` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:03:57.285683 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 653/1312] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jul 13 22:03:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 22:03:57.291665 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c27755-a829-43ab-b519-cac1d898f9a9 None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:03:57.292127 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c27755-a829-43ab-b519-cac1d898f9a9 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:03:57.292219 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7c27755-a829-43ab-b519-cac1d898f9a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:03:57.369606 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-a7c27755-a829-43ab-b519-cac1d898f9a9 None None] MFA Rules not processed for user `cc99c93da5074cf5a48bc2aa9a92c5e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 22:03:57.370864 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-a7c27755-a829-43ab-b519-cac1d898f9a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:03:57.480591 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 660/1313] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jul 13 22:03:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 22:03:57.518419 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c6c720-a19c-42f7-ba8f-cedf2f80d3ae None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:03:57.520028 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-a5c6c720-a19c-42f7-ba8f-cedf2f80d3ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:03:57.637341 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5c6c720-a19c-42f7-ba8f-cedf2f80d3ae None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:03:57.715459 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:03:57.718145 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d2d2ccc-c76d-4c32-b687-51c7ca5e81eb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '847ec9a1bd4d4dec804318a5c845895a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417e3e2f1333481795a089bef2e24bea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:03:57.718915 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2d2ccc-c76d-4c32-b687-51c7ca5e81eb service neutron] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:03:57.719187 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2d2ccc-c76d-4c32-b687-51c7ca5e81eb service neutron] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:03:57.719451 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d2d2ccc-c76d-4c32-b687-51c7ca5e81eb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:03:57.721942 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-8d2d2ccc-c76d-4c32-b687-51c7ca5e81eb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:03:57.841910 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2d2ccc-c76d-4c32-b687-51c7ca5e81eb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:03:57.843397 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d2d2ccc-c76d-4c32-b687-51c7ca5e81eb service neutron] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:03:57.862863 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 654/1314] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jul 13 22:03:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 22:04:00.248832 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a641ba-97d9-4889-93bb-5c3716884594 None None] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:04:00.249429 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a641ba-97d9-4889-93bb-5c3716884594 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:04:00.249429 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-70a641ba-97d9-4889-93bb-5c3716884594 None None] PATH_INFO: `/` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:04:00.250980 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 661/1315] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jul 13 22:04:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 22:04:00.256816 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5460a6-49fd-466d-8ff8-3b9ba659b3f7 None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:04:00.258018 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5460a6-49fd-466d-8ff8-3b9ba659b3f7 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:04:00.258018 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e5460a6-49fd-466d-8ff8-3b9ba659b3f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:04:00.328176 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-1e5460a6-49fd-466d-8ff8-3b9ba659b3f7 None None] MFA Rules not processed for user `4041442c17b840b697a19b97785daf24`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 22:04:00.329400 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-1e5460a6-49fd-466d-8ff8-3b9ba659b3f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:04:00.380906 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:04:00.428819 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 655/1316] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jul 13 22:04:00 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 22:04:00.481495 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f825ed97-ff2a-4d1a-935e-a48ce6929009 None None] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:04:00.481965 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f825ed97-ff2a-4d1a-935e-a48ce6929009 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:04:00.482304 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f825ed97-ff2a-4d1a-935e-a48ce6929009 None None] PATH_INFO: `/` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:04:00.483803 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 662/1317] 10.222.0.52 () {54 vars in 964 bytes} [Wed Jul 13 22:04:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 22:04:00.489914 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd60ba80-fa9b-4dde-bded-fab22aa0dcc6 None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:04:00.490204 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd60ba80-fa9b-4dde-bded-fab22aa0dcc6 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:04:00.490480 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd60ba80-fa9b-4dde-bded-fab22aa0dcc6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:04:00.573751 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-bd60ba80-fa9b-4dde-bded-fab22aa0dcc6 None None] MFA Rules not processed for user `847ec9a1bd4d4dec804318a5c845895a`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 22:04:00.575013 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-bd60ba80-fa9b-4dde-bded-fab22aa0dcc6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:04:00.656224 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 656/1318] 10.222.0.52 () {58 vars in 1061 bytes} [Wed Jul 13 22:04:00 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 13 22:04:00.668605 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f825ed97-ff2a-4d1a-935e-a48ce6929009 None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:04:00.671048 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-f825ed97-ff2a-4d1a-935e-a48ce6929009 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:04:00.686804 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:04:00.788722 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f825ed97-ff2a-4d1a-935e-a48ce6929009 None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:04:00.876933 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ff14100-0352-4b65-bdf8-8bd08ff98634 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '847ec9a1bd4d4dec804318a5c845895a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417e3e2f1333481795a089bef2e24bea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:04:00.877890 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff14100-0352-4b65-bdf8-8bd08ff98634 service neutron] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:04:00.878288 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff14100-0352-4b65-bdf8-8bd08ff98634 service neutron] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:04:00.878707 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ff14100-0352-4b65-bdf8-8bd08ff98634 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:04:00.881333 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-0ff14100-0352-4b65-bdf8-8bd08ff98634 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:04:00.953029 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:04:01.002331 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff14100-0352-4b65-bdf8-8bd08ff98634 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:04:01.003823 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ff14100-0352-4b65-bdf8-8bd08ff98634 service neutron] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:04:01.022688 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 663/1319] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jul 13 22:04:00 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 355 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 22:04:04.003395 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ec60f5-753d-48e7-80ad-d017f26d6596 None None] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:04:04.004271 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ec60f5-753d-48e7-80ad-d017f26d6596 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:04:04.004271 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56ec60f5-753d-48e7-80ad-d017f26d6596 None None] PATH_INFO: `/` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:04:04.006470 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 657/1320] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jul 13 22:04:04 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 22:04:04.012896 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eec7f57-06f8-4e48-a8b1-256e07efd366 None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:04:04.013355 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eec7f57-06f8-4e48-a8b1-256e07efd366 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:04:04.013586 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3eec7f57-06f8-4e48-a8b1-256e07efd366 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:04:04.082256 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-3eec7f57-06f8-4e48-a8b1-256e07efd366 None None] MFA Rules not processed for user `cc99c93da5074cf5a48bc2aa9a92c5e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 22:04:04.083502 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-3eec7f57-06f8-4e48-a8b1-256e07efd366 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:04:04.191602 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 664/1321] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jul 13 22:04:04 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 22:04:04.230849 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ec60f5-753d-48e7-80ad-d017f26d6596 None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:04:04.232397 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-56ec60f5-753d-48e7-80ad-d017f26d6596 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:04:04.288487 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:04:04.349268 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56ec60f5-753d-48e7-80ad-d017f26d6596 None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:04:04.430092 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41eae836-e163-4afc-b5cb-81dc171d9849 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '847ec9a1bd4d4dec804318a5c845895a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417e3e2f1333481795a089bef2e24bea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:04:04.430779 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41eae836-e163-4afc-b5cb-81dc171d9849 service neutron] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:04:04.431056 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41eae836-e163-4afc-b5cb-81dc171d9849 service neutron] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:04:04.431327 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41eae836-e163-4afc-b5cb-81dc171d9849 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:04:04.433002 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-41eae836-e163-4afc-b5cb-81dc171d9849 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:04:04.539047 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:04:04.559732 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41eae836-e163-4afc-b5cb-81dc171d9849 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:04:04.561190 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41eae836-e163-4afc-b5cb-81dc171d9849 service neutron] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:04:04.582184 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 658/1322] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jul 13 22:04:04 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 352 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 22:04:07.307986 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1503dca2-8fec-4b00-9782-bd08ed853509 None None] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:04:07.308562 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1503dca2-8fec-4b00-9782-bd08ed853509 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:04:07.308562 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1503dca2-8fec-4b00-9782-bd08ed853509 None None] PATH_INFO: `/` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:04:07.309904 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 665/1323] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jul 13 22:04:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 22:04:07.315323 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03e5186-52db-4422-8ed7-203036715447 None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:04:07.315912 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03e5186-52db-4422-8ed7-203036715447 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:04:07.315912 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03e5186-52db-4422-8ed7-203036715447 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:04:07.381643 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-c03e5186-52db-4422-8ed7-203036715447 None None] MFA Rules not processed for user `cc99c93da5074cf5a48bc2aa9a92c5e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 22:04:07.382976 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-c03e5186-52db-4422-8ed7-203036715447 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:04:07.487140 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 659/1324] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jul 13 22:04:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 173 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 22:04:07.523588 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1503dca2-8fec-4b00-9782-bd08ed853509 None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:04:07.525066 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-1503dca2-8fec-4b00-9782-bd08ed853509 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:04:07.546602 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:04:07.643867 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1503dca2-8fec-4b00-9782-bd08ed853509 None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:04:07.718470 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8492d079-d4c2-4e04-87b9-2b449505e46f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '847ec9a1bd4d4dec804318a5c845895a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417e3e2f1333481795a089bef2e24bea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:04:07.719247 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8492d079-d4c2-4e04-87b9-2b449505e46f service neutron] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:04:07.719526 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8492d079-d4c2-4e04-87b9-2b449505e46f service neutron] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:04:07.719792 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8492d079-d4c2-4e04-87b9-2b449505e46f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:04:07.721448 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-8492d079-d4c2-4e04-87b9-2b449505e46f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:04:07.802878 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:04:07.839332 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8492d079-d4c2-4e04-87b9-2b449505e46f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:04:07.840794 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8492d079-d4c2-4e04-87b9-2b449505e46f service neutron] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:04:07.861080 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 666/1325] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jul 13 22:04:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 338 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 22:04:09.928475 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3430b8-dab2-45a2-968b-739ce5ff4a91 None None] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:04:09.929115 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3430b8-dab2-45a2-968b-739ce5ff4a91 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:04:09.929115 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a3430b8-dab2-45a2-968b-739ce5ff4a91 None None] PATH_INFO: `/` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:04:09.930590 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 660/1326] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jul 13 22:04:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 22:04:09.935965 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb7ebaf-dd62-45ff-bd7f-5385985a3541 None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:04:09.936445 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb7ebaf-dd62-45ff-bd7f-5385985a3541 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:04:09.936541 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fb7ebaf-dd62-45ff-bd7f-5385985a3541 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:04:10.001902 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-1fb7ebaf-dd62-45ff-bd7f-5385985a3541 None None] MFA Rules not processed for user `cc99c93da5074cf5a48bc2aa9a92c5e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 22:04:10.003136 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-1fb7ebaf-dd62-45ff-bd7f-5385985a3541 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:04:10.106207 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 667/1327] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jul 13 22:04:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 171 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 22:04:10.142221 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3430b8-dab2-45a2-968b-739ce5ff4a91 None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:04:10.143822 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-9a3430b8-dab2-45a2-968b-739ce5ff4a91 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:04:10.206725 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:04:10.252694 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a3430b8-dab2-45a2-968b-739ce5ff4a91 None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:04:10.337320 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e1ce00a-c63a-416c-bcee-f4498e906cfd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '847ec9a1bd4d4dec804318a5c845895a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417e3e2f1333481795a089bef2e24bea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:04:10.337972 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1ce00a-c63a-416c-bcee-f4498e906cfd service neutron] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:04:10.338247 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1ce00a-c63a-416c-bcee-f4498e906cfd service neutron] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:04:10.338540 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e1ce00a-c63a-416c-bcee-f4498e906cfd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:04:10.340191 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-2e1ce00a-c63a-416c-bcee-f4498e906cfd service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:04:10.450332 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:04:10.453288 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1ce00a-c63a-416c-bcee-f4498e906cfd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:04:10.454992 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e1ce00a-c63a-416c-bcee-f4498e906cfd service neutron] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:04:10.482837 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 661/1328] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jul 13 22:04:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 22:04:12.552828 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8cfa60-3c8b-43c0-9654-d085c4292643 None None] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:04:12.553338 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8cfa60-3c8b-43c0-9654-d085c4292643 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:04:12.553338 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae8cfa60-3c8b-43c0-9654-d085c4292643 None None] PATH_INFO: `/` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:04:12.555344 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 668/1329] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jul 13 22:04:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 22:04:12.560795 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab66bcc4-1447-449b-9b4c-deb280c63db8 None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:04:12.561282 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab66bcc4-1447-449b-9b4c-deb280c63db8 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:04:12.561370 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab66bcc4-1447-449b-9b4c-deb280c63db8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:04:12.630456 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-ab66bcc4-1447-449b-9b4c-deb280c63db8 None None] MFA Rules not processed for user `4041442c17b840b697a19b97785daf24`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 22:04:12.631775 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-ab66bcc4-1447-449b-9b4c-deb280c63db8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:04:12.734352 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 662/1330] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jul 13 22:04:12 2022] POST /identity/v3/auth/tokens => generated 3233 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 22:04:12.771859 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8cfa60-3c8b-43c0-9654-d085c4292643 None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:04:12.773550 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-ae8cfa60-3c8b-43c0-9654-d085c4292643 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:04:12.806206 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:04:12.882543 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae8cfa60-3c8b-43c0-9654-d085c4292643 None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:04:12.958856 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-094ac20e-6ba2-45c4-aa37-6bc72a2d99fb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '847ec9a1bd4d4dec804318a5c845895a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417e3e2f1333481795a089bef2e24bea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:04:12.959509 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094ac20e-6ba2-45c4-aa37-6bc72a2d99fb service neutron] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:04:12.959876 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094ac20e-6ba2-45c4-aa37-6bc72a2d99fb service neutron] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:04:12.960229 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-094ac20e-6ba2-45c4-aa37-6bc72a2d99fb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:04:12.961867 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-094ac20e-6ba2-45c4-aa37-6bc72a2d99fb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:04:13.038076 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:04:13.091847 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094ac20e-6ba2-45c4-aa37-6bc72a2d99fb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:04:13.093303 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-094ac20e-6ba2-45c4-aa37-6bc72a2d99fb service neutron] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:04:13.115101 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 669/1331] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jul 13 22:04:12 2022] GET /identity/v3/auth/tokens => generated 3233 bytes in 344 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 22:04:15.404894 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93854c2-410d-449f-9325-b84e559209c2 None None] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:04:15.405847 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93854c2-410d-449f-9325-b84e559209c2 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:04:15.405847 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93854c2-410d-449f-9325-b84e559209c2 None None] PATH_INFO: `/` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:04:15.408112 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 663/1332] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jul 13 22:04:15 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 22:04:15.415901 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6375c42d-5780-4a45-8208-45097093776b None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:04:15.416324 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6375c42d-5780-4a45-8208-45097093776b None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:04:15.416470 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6375c42d-5780-4a45-8208-45097093776b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:04:15.485822 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-6375c42d-5780-4a45-8208-45097093776b None None] MFA Rules not processed for user `cc99c93da5074cf5a48bc2aa9a92c5e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 22:04:15.486965 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-6375c42d-5780-4a45-8208-45097093776b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:04:15.586876 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 670/1333] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jul 13 22:04:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 172 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 22:04:15.627527 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93854c2-410d-449f-9325-b84e559209c2 None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:04:15.629346 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-c93854c2-410d-449f-9325-b84e559209c2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:04:15.708165 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:04:15.748657 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93854c2-410d-449f-9325-b84e559209c2 None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:04:15.837525 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbd36db0-35b1-4ab7-9464-1a0e5bcda8ff service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '847ec9a1bd4d4dec804318a5c845895a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417e3e2f1333481795a089bef2e24bea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:04:15.838189 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd36db0-35b1-4ab7-9464-1a0e5bcda8ff service neutron] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:04:15.838476 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd36db0-35b1-4ab7-9464-1a0e5bcda8ff service neutron] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:04:15.838779 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbd36db0-35b1-4ab7-9464-1a0e5bcda8ff service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:04:15.840462 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-dbd36db0-35b1-4ab7-9464-1a0e5bcda8ff service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:04:15.951383 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd36db0-35b1-4ab7-9464-1a0e5bcda8ff service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:04:15.952830 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbd36db0-35b1-4ab7-9464-1a0e5bcda8ff service neutron] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:04:15.962876 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:04:15.971767 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 664/1334] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jul 13 22:04:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 22:04:18.039218 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e56f30-1ed6-4e23-bf69-f78648d05c3c None None] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:04:18.040122 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e56f30-1ed6-4e23-bf69-f78648d05c3c None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:04:18.040122 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31e56f30-1ed6-4e23-bf69-f78648d05c3c None None] PATH_INFO: `/` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:04:18.042327 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 671/1335] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jul 13 22:04:18 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 22:04:18.047868 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da106c4-e79a-403a-9700-97b0b2573049 None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:04:18.048306 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da106c4-e79a-403a-9700-97b0b2573049 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:04:18.048431 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3da106c4-e79a-403a-9700-97b0b2573049 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:04:18.117512 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-3da106c4-e79a-403a-9700-97b0b2573049 None None] MFA Rules not processed for user `cc99c93da5074cf5a48bc2aa9a92c5e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 22:04:18.118687 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-3da106c4-e79a-403a-9700-97b0b2573049 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:04:18.222370 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 665/1336] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jul 13 22:04:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 22:04:18.265659 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e56f30-1ed6-4e23-bf69-f78648d05c3c None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:04:18.268174 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-31e56f30-1ed6-4e23-bf69-f78648d05c3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:04:18.317077 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:04:18.384904 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31e56f30-1ed6-4e23-bf69-f78648d05c3c None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:04:18.473021 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b93d95-39b8-4ec4-81dc-947e68e31ea8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '847ec9a1bd4d4dec804318a5c845895a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417e3e2f1333481795a089bef2e24bea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:04:18.473665 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b93d95-39b8-4ec4-81dc-947e68e31ea8 service neutron] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:04:18.473950 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b93d95-39b8-4ec4-81dc-947e68e31ea8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:04:18.474217 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b93d95-39b8-4ec4-81dc-947e68e31ea8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:04:18.475945 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-77b93d95-39b8-4ec4-81dc-947e68e31ea8 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:04:18.560321 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:04:18.588241 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b93d95-39b8-4ec4-81dc-947e68e31ea8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:04:18.589714 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b93d95-39b8-4ec4-81dc-947e68e31ea8 service neutron] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:04:18.609607 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 672/1337] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jul 13 22:04:18 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 22:04:21.081527 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80eae69-3d2f-4f29-a6ab-ea00950ca43f None None] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:04:21.082407 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80eae69-3d2f-4f29-a6ab-ea00950ca43f None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:04:21.082589 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b80eae69-3d2f-4f29-a6ab-ea00950ca43f None None] PATH_INFO: `/` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:04:21.084945 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 666/1338] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jul 13 22:04:21 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 22:04:21.091655 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b2e693-6f14-4b15-a632-c99184909648 None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:04:21.092368 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b2e693-6f14-4b15-a632-c99184909648 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:04:21.092500 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67b2e693-6f14-4b15-a632-c99184909648 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:04:21.168752 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-67b2e693-6f14-4b15-a632-c99184909648 None None] MFA Rules not processed for user `cc99c93da5074cf5a48bc2aa9a92c5e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 22:04:21.169943 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-67b2e693-6f14-4b15-a632-c99184909648 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:04:21.269598 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 673/1339] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jul 13 22:04:21 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 13 22:04:21.316830 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80eae69-3d2f-4f29-a6ab-ea00950ca43f None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:04:21.318762 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-b80eae69-3d2f-4f29-a6ab-ea00950ca43f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:04:21.406628 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:04:21.436105 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b80eae69-3d2f-4f29-a6ab-ea00950ca43f None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:04:21.511367 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9330bbef-3661-4aec-aeb0-fa31606c5b3e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '847ec9a1bd4d4dec804318a5c845895a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417e3e2f1333481795a089bef2e24bea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:04:21.512013 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9330bbef-3661-4aec-aeb0-fa31606c5b3e service neutron] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:04:21.512306 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9330bbef-3661-4aec-aeb0-fa31606c5b3e service neutron] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:04:21.512549 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9330bbef-3661-4aec-aeb0-fa31606c5b3e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:04:21.514187 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-9330bbef-3661-4aec-aeb0-fa31606c5b3e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:04:21.633822 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9330bbef-3661-4aec-aeb0-fa31606c5b3e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:04:21.636485 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9330bbef-3661-4aec-aeb0-fa31606c5b3e service neutron] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:04:21.657208 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 667/1340] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jul 13 22:04:21 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 22:04:25.650292 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bff58d5-c2f3-4f6e-89f1-2e170b8bb35e None None] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:04:25.650883 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bff58d5-c2f3-4f6e-89f1-2e170b8bb35e None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:04:25.650883 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2bff58d5-c2f3-4f6e-89f1-2e170b8bb35e None None] PATH_INFO: `/` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:04:25.652277 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 674/1341] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jul 13 22:04:25 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 22:04:25.658101 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540e67ff-92ed-4935-8e39-a4f25d4d6d7e None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:04:25.658585 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540e67ff-92ed-4935-8e39-a4f25d4d6d7e None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:04:25.658825 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-540e67ff-92ed-4935-8e39-a4f25d4d6d7e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:04:25.663235 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:04:25.728999 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-540e67ff-92ed-4935-8e39-a4f25d4d6d7e None None] MFA Rules not processed for user `cc99c93da5074cf5a48bc2aa9a92c5e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 22:04:25.730200 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-540e67ff-92ed-4935-8e39-a4f25d4d6d7e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:04:25.831907 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 668/1342] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jul 13 22:04:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 22:04:25.870128 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bff58d5-c2f3-4f6e-89f1-2e170b8bb35e None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:04:25.871714 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-2bff58d5-c2f3-4f6e-89f1-2e170b8bb35e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:04:25.921498 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:04:25.983307 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2bff58d5-c2f3-4f6e-89f1-2e170b8bb35e None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:04:26.056595 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be3e1695-b596-4a77-888f-378d7c8edc6a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '847ec9a1bd4d4dec804318a5c845895a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417e3e2f1333481795a089bef2e24bea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:04:26.057368 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3e1695-b596-4a77-888f-378d7c8edc6a service neutron] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:04:26.057686 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3e1695-b596-4a77-888f-378d7c8edc6a service neutron] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:04:26.057980 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be3e1695-b596-4a77-888f-378d7c8edc6a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:04:26.059681 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-be3e1695-b596-4a77-888f-378d7c8edc6a service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:04:26.157576 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:04:26.175165 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3e1695-b596-4a77-888f-378d7c8edc6a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:04:26.176533 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be3e1695-b596-4a77-888f-378d7c8edc6a service neutron] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:04:26.195616 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 675/1343] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jul 13 22:04:25 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 326 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 22:04:28.892101 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660bc004-4c4e-45f0-b64d-f5ec1aa4976c None None] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:04:28.892700 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660bc004-4c4e-45f0-b64d-f5ec1aa4976c None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:04:28.892700 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-660bc004-4c4e-45f0-b64d-f5ec1aa4976c None None] PATH_INFO: `/` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:04:28.893968 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 669/1344] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jul 13 22:04:28 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 22:04:28.900291 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b50ae58-daf8-4a9f-b917-1d6105e135f0 None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:04:28.900897 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b50ae58-daf8-4a9f-b917-1d6105e135f0 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:04:28.900897 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b50ae58-daf8-4a9f-b917-1d6105e135f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:04:28.972939 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-9b50ae58-daf8-4a9f-b917-1d6105e135f0 None None] MFA Rules not processed for user `cc99c93da5074cf5a48bc2aa9a92c5e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 22:04:28.974158 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-9b50ae58-daf8-4a9f-b917-1d6105e135f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:04:29.088788 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 676/1345] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jul 13 22:04:28 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 13 22:04:29.126933 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660bc004-4c4e-45f0-b64d-f5ec1aa4976c None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:04:29.128484 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-660bc004-4c4e-45f0-b64d-f5ec1aa4976c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:04:29.219657 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:04:29.237382 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-660bc004-4c4e-45f0-b64d-f5ec1aa4976c None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:04:29.314345 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6d34be-2dff-4ad7-bc1a-8b62ceede62d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '847ec9a1bd4d4dec804318a5c845895a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417e3e2f1333481795a089bef2e24bea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:04:29.315043 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6d34be-2dff-4ad7-bc1a-8b62ceede62d service neutron] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:04:29.315299 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6d34be-2dff-4ad7-bc1a-8b62ceede62d service neutron] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:04:29.315749 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6d34be-2dff-4ad7-bc1a-8b62ceede62d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:04:29.317515 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-9a6d34be-2dff-4ad7-bc1a-8b62ceede62d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:04:29.434013 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6d34be-2dff-4ad7-bc1a-8b62ceede62d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:04:29.435490 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9a6d34be-2dff-4ad7-bc1a-8b62ceede62d service neutron] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:04:29.454553 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 670/1346] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jul 13 22:04:29 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 328 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 22:04:33.557016 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e704136-9fea-4b30-8472-45cd531f5434 None None] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:04:33.557549 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e704136-9fea-4b30-8472-45cd531f5434 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:04:33.557549 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e704136-9fea-4b30-8472-45cd531f5434 None None] PATH_INFO: `/` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:04:33.558939 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 677/1347] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jul 13 22:04:33 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 22:04:33.564477 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5707b079-711a-48d1-b4bf-3b3ab96b0874 None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:04:33.564913 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5707b079-711a-48d1-b4bf-3b3ab96b0874 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:04:33.565019 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5707b079-711a-48d1-b4bf-3b3ab96b0874 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:04:33.581788 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:04:33.655018 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-5707b079-711a-48d1-b4bf-3b3ab96b0874 None None] MFA Rules not processed for user `cc99c93da5074cf5a48bc2aa9a92c5e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 22:04:33.656276 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-5707b079-711a-48d1-b4bf-3b3ab96b0874 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:04:33.763403 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 671/1348] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jul 13 22:04:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 200 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 22:04:33.802101 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e704136-9fea-4b30-8472-45cd531f5434 None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:04:33.803879 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-3e704136-9fea-4b30-8472-45cd531f5434 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:04:33.875690 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:04:33.930766 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e704136-9fea-4b30-8472-45cd531f5434 None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:04:34.019856 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de52240c-217f-4d83-a37c-aae844e6a34c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '847ec9a1bd4d4dec804318a5c845895a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417e3e2f1333481795a089bef2e24bea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:04:34.020522 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de52240c-217f-4d83-a37c-aae844e6a34c service neutron] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:04:34.020803 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de52240c-217f-4d83-a37c-aae844e6a34c service neutron] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:04:34.021074 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de52240c-217f-4d83-a37c-aae844e6a34c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:04:34.022912 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-de52240c-217f-4d83-a37c-aae844e6a34c service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:04:34.142635 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:04:34.146398 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de52240c-217f-4d83-a37c-aae844e6a34c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:04:34.148513 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de52240c-217f-4d83-a37c-aae844e6a34c service neutron] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:04:34.167712 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 678/1349] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jul 13 22:04:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 22:04:39.094180 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ff4d2c-4b3e-42af-bc66-547ca7e66d3c None None] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:04:39.094739 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ff4d2c-4b3e-42af-bc66-547ca7e66d3c None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:04:39.094739 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43ff4d2c-4b3e-42af-bc66-547ca7e66d3c None None] PATH_INFO: `/` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:04:39.096154 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 672/1350] 10.222.0.52 () {54 vars in 921 bytes} [Wed Jul 13 22:04:39 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 22:04:39.101230 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d59e63c-cf15-4cb8-99b1-c9f45c6b2cdf None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:04:39.101677 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d59e63c-cf15-4cb8-99b1-c9f45c6b2cdf None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:04:39.101775 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d59e63c-cf15-4cb8-99b1-c9f45c6b2cdf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:04:39.213868 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-1d59e63c-cf15-4cb8-99b1-c9f45c6b2cdf None None] MFA Rules not processed for user `ac673d641d8849c79bf4cf71c9805587`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 22:04:39.215347 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-1d59e63c-cf15-4cb8-99b1-c9f45c6b2cdf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:04:39.357095 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 679/1351] 10.222.0.52 () {58 vars in 1018 bytes} [Wed Jul 13 22:04:39 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 246 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 22:04:43.616510 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e8158a-a2bb-4ac3-b18f-dd4210e303ca None None] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:04:43.617481 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e8158a-a2bb-4ac3-b18f-dd4210e303ca None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:04:43.617748 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39e8158a-a2bb-4ac3-b18f-dd4210e303ca None None] PATH_INFO: `/` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:04:43.619347 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 673/1352] 10.222.0.52 () {54 vars in 921 bytes} [Wed Jul 13 22:04:43 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 22:04:43.626514 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2467f0d-21c6-41c3-ab85-653071fd1436 None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:04:43.626982 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2467f0d-21c6-41c3-ab85-653071fd1436 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:04:43.626982 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2467f0d-21c6-41c3-ab85-653071fd1436 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:04:43.683003 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:04:43.711657 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-b2467f0d-21c6-41c3-ab85-653071fd1436 None None] MFA Rules not processed for user `ac673d641d8849c79bf4cf71c9805587`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 22:04:43.712880 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-b2467f0d-21c6-41c3-ab85-653071fd1436 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:04:43.762842 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=59 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:04:43.837800 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 680/1353] 10.222.0.52 () {58 vars in 1018 bytes} [Wed Jul 13 22:04:43 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 212 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 22:04:45.033303 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf02f613-43d1-492a-90ca-d26802e02cb6 None None] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:04:45.033864 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf02f613-43d1-492a-90ca-d26802e02cb6 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:04:45.033864 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf02f613-43d1-492a-90ca-d26802e02cb6 None None] PATH_INFO: `/` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:04:45.035265 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 674/1354] 10.222.0.52 () {54 vars in 921 bytes} [Wed Jul 13 22:04:45 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 22:04:45.040766 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf244f4f-84bb-4531-893e-1682f7a581ee None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:04:45.041236 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf244f4f-84bb-4531-893e-1682f7a581ee None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:04:45.041450 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf244f4f-84bb-4531-893e-1682f7a581ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:04:45.120015 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-bf244f4f-84bb-4531-893e-1682f7a581ee None None] MFA Rules not processed for user `ac673d641d8849c79bf4cf71c9805587`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 22:04:45.121207 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-bf244f4f-84bb-4531-893e-1682f7a581ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:04:45.164516 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:04:45.233066 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 681/1355] 10.222.0.52 () {58 vars in 1018 bytes} [Wed Jul 13 22:04:45 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 13 22:04:48.715536 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d909ae35-bd7f-435f-a5a1-4eb486f21d83 None None] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:04:48.716078 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d909ae35-bd7f-435f-a5a1-4eb486f21d83 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:04:48.716078 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d909ae35-bd7f-435f-a5a1-4eb486f21d83 None None] PATH_INFO: `/` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:04:48.717396 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 675/1356] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jul 13 22:04:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 22:04:48.722576 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2a1e27-6111-4e0f-9ba8-7a18cc0e6124 None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:04:48.723006 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2a1e27-6111-4e0f-9ba8-7a18cc0e6124 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:04:48.723208 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f2a1e27-6111-4e0f-9ba8-7a18cc0e6124 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:04:48.797122 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-1f2a1e27-6111-4e0f-9ba8-7a18cc0e6124 None None] MFA Rules not processed for user `cc99c93da5074cf5a48bc2aa9a92c5e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 22:04:48.798339 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-1f2a1e27-6111-4e0f-9ba8-7a18cc0e6124 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:04:48.902384 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 682/1357] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jul 13 22:04:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 22:04:48.959815 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8514f5-7301-439f-bd5a-6819511e4f47 None None] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:04:48.960395 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8514f5-7301-439f-bd5a-6819511e4f47 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:04:48.960879 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf8514f5-7301-439f-bd5a-6819511e4f47 None None] PATH_INFO: `/` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:04:48.962772 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 676/1358] 10.222.0.52 () {54 vars in 961 bytes} [Wed Jul 13 22:04:48 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 22:04:48.970182 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83da5ba5-33b1-4995-8403-23af9c11e469 None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:04:48.970817 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83da5ba5-33b1-4995-8403-23af9c11e469 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:04:48.971319 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83da5ba5-33b1-4995-8403-23af9c11e469 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:04:48.990286 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:04:49.055095 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-83da5ba5-33b1-4995-8403-23af9c11e469 None None] MFA Rules not processed for user `f84281091687464990ffbca590b9343b`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 22:04:49.056425 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-83da5ba5-33b1-4995-8403-23af9c11e469 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:04:49.176046 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 683/1359] 10.222.0.52 () {58 vars in 1058 bytes} [Wed Jul 13 22:04:48 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 22:04:49.183860 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8514f5-7301-439f-bd5a-6819511e4f47 None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:04:49.185381 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-cf8514f5-7301-439f-bd5a-6819511e4f47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:04:49.359342 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf8514f5-7301-439f-bd5a-6819511e4f47 None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:04:49.437367 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:04:49.480282 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92a17a74-3f75-491c-ad55-efddc9440e77 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84281091687464990ffbca590b9343b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417e3e2f1333481795a089bef2e24bea', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:04:49.481039 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a17a74-3f75-491c-ad55-efddc9440e77 service nova] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:04:49.481325 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a17a74-3f75-491c-ad55-efddc9440e77 service nova] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:04:49.481599 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a17a74-3f75-491c-ad55-efddc9440e77 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:04:49.483268 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-92a17a74-3f75-491c-ad55-efddc9440e77 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:04:49.586057 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a17a74-3f75-491c-ad55-efddc9440e77 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:04:49.588061 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-92a17a74-3f75-491c-ad55-efddc9440e77 service nova] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:04:49.608966 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 677/1360] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jul 13 22:04:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 425 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 22:04:51.854854 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59f0bef-5342-4320-b686-f89e616c2a1c None None] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:04:51.855386 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59f0bef-5342-4320-b686-f89e616c2a1c None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:04:51.855386 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d59f0bef-5342-4320-b686-f89e616c2a1c None None] PATH_INFO: `/` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:04:51.856951 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 684/1361] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jul 13 22:04:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 22:04:51.862201 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c586729-3cf2-45af-81c7-43016c182230 None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:04:51.862655 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c586729-3cf2-45af-81c7-43016c182230 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:04:51.862858 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c586729-3cf2-45af-81c7-43016c182230 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:04:51.932726 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-1c586729-3cf2-45af-81c7-43016c182230 None None] MFA Rules not processed for user `cc99c93da5074cf5a48bc2aa9a92c5e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 22:04:51.933991 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-1c586729-3cf2-45af-81c7-43016c182230 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:04:51.978145 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:04:52.053159 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 678/1362] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jul 13 22:04:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 22:04:52.101514 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59f0bef-5342-4320-b686-f89e616c2a1c None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:04:52.102974 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-d59f0bef-5342-4320-b686-f89e616c2a1c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:04:52.182097 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:04:52.303420 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d59f0bef-5342-4320-b686-f89e616c2a1c None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:04:52.423985 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-733efff1-78b1-4428-b971-5f02c95e3b97 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84281091687464990ffbca590b9343b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417e3e2f1333481795a089bef2e24bea', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:04:52.424644 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733efff1-78b1-4428-b971-5f02c95e3b97 service nova] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:04:52.424927 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733efff1-78b1-4428-b971-5f02c95e3b97 service nova] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:04:52.425189 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-733efff1-78b1-4428-b971-5f02c95e3b97 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:04:52.426906 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-733efff1-78b1-4428-b971-5f02c95e3b97 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:04:52.450897 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:04:52.520540 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733efff1-78b1-4428-b971-5f02c95e3b97 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:04:52.522253 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-733efff1-78b1-4428-b971-5f02c95e3b97 service nova] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:04:52.543907 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 685/1363] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jul 13 22:04:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 443 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 22:04:54.945530 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999d0286-351b-40b0-93ab-c8c7d0b948ff None None] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:04:54.946084 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999d0286-351b-40b0-93ab-c8c7d0b948ff None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:04:54.946084 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-999d0286-351b-40b0-93ab-c8c7d0b948ff None None] PATH_INFO: `/` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:04:54.947595 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 679/1364] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jul 13 22:04:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 22:04:54.953409 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb308a3-8fa1-4ae9-b334-14e4b14e86c5 None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:04:54.953883 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb308a3-8fa1-4ae9-b334-14e4b14e86c5 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:04:54.954247 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1eb308a3-8fa1-4ae9-b334-14e4b14e86c5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:04:55.024447 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-1eb308a3-8fa1-4ae9-b334-14e4b14e86c5 None None] MFA Rules not processed for user `cc99c93da5074cf5a48bc2aa9a92c5e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 22:04:55.025812 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-1eb308a3-8fa1-4ae9-b334-14e4b14e86c5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:04:55.126176 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:04:55.134628 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 686/1365] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jul 13 22:04:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 22:04:55.173610 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2233c5e9-8acb-468d-b256-1a7f71dc9624 None None] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:04:55.174137 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2233c5e9-8acb-468d-b256-1a7f71dc9624 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:04:55.174422 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2233c5e9-8acb-468d-b256-1a7f71dc9624 None None] PATH_INFO: `/` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:04:55.176013 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 680/1366] 10.222.0.52 () {54 vars in 961 bytes} [Wed Jul 13 22:04:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 22:04:55.181934 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a589a6-b06d-4be8-abed-41fbbe59449a None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:04:55.182292 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a589a6-b06d-4be8-abed-41fbbe59449a None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:04:55.182658 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5a589a6-b06d-4be8-abed-41fbbe59449a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:04:55.262918 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-a5a589a6-b06d-4be8-abed-41fbbe59449a None None] MFA Rules not processed for user `f84281091687464990ffbca590b9343b`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 22:04:55.264075 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-a5a589a6-b06d-4be8-abed-41fbbe59449a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:04:55.396607 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 687/1367] 10.222.0.52 () {58 vars in 1058 bytes} [Wed Jul 13 22:04:55 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 216 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 22:04:55.405848 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2233c5e9-8acb-468d-b256-1a7f71dc9624 None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:04:55.407415 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-2233c5e9-8acb-468d-b256-1a7f71dc9624 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:04:55.588736 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2233c5e9-8acb-468d-b256-1a7f71dc9624 None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:04:55.610859 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:04:55.716358 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64be5f6f-4c32-436b-8fa9-fd7e89d339f7 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84281091687464990ffbca590b9343b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417e3e2f1333481795a089bef2e24bea', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:04:55.717022 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64be5f6f-4c32-436b-8fa9-fd7e89d339f7 service nova] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:04:55.717355 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64be5f6f-4c32-436b-8fa9-fd7e89d339f7 service nova] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:04:55.717632 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64be5f6f-4c32-436b-8fa9-fd7e89d339f7 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:04:55.719320 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-64be5f6f-4c32-436b-8fa9-fd7e89d339f7 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:04:55.824649 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64be5f6f-4c32-436b-8fa9-fd7e89d339f7 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:04:55.826210 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64be5f6f-4c32-436b-8fa9-fd7e89d339f7 service nova] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:04:55.845150 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 681/1368] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jul 13 22:04:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 440 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 22:04:56.657337 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658d6f40-ad9c-4889-813f-9a7ba4414c6c None None] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:04:56.657899 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658d6f40-ad9c-4889-813f-9a7ba4414c6c None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:04:56.657899 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-658d6f40-ad9c-4889-813f-9a7ba4414c6c None None] PATH_INFO: `/` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:04:56.659407 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 688/1369] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jul 13 22:04:56 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 22:04:56.664934 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f86a120-f665-4f39-8ce5-e8467b0cedb0 None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:04:56.665494 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f86a120-f665-4f39-8ce5-e8467b0cedb0 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:04:56.665494 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f86a120-f665-4f39-8ce5-e8467b0cedb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:04:56.698741 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:04:56.739249 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-3f86a120-f665-4f39-8ce5-e8467b0cedb0 None None] MFA Rules not processed for user `cc99c93da5074cf5a48bc2aa9a92c5e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 22:04:56.740485 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-3f86a120-f665-4f39-8ce5-e8467b0cedb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:04:56.841501 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 682/1370] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jul 13 22:04:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 22:04:57.178750 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53dda91d-1cba-4026-8331-eccfee13427e None None] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:04:57.179147 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53dda91d-1cba-4026-8331-eccfee13427e None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:04:57.179325 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53dda91d-1cba-4026-8331-eccfee13427e None None] PATH_INFO: `/` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:04:57.180827 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 689/1371] 10.222.0.52 () {54 vars in 963 bytes} [Wed Jul 13 22:04:57 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 22:04:57.185751 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfe4a95-6ef1-4420-97ef-11a00d517ebf None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:04:57.186011 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfe4a95-6ef1-4420-97ef-11a00d517ebf None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:04:57.186311 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dfe4a95-6ef1-4420-97ef-11a00d517ebf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:04:57.265507 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-9dfe4a95-6ef1-4420-97ef-11a00d517ebf None None] MFA Rules not processed for user `bc77e1ea1ada4700b809beb1c980c5e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 22:04:57.266737 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-9dfe4a95-6ef1-4420-97ef-11a00d517ebf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:04:57.320525 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:04:57.379508 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 683/1372] 10.222.0.52 () {58 vars in 1060 bytes} [Wed Jul 13 22:04:57 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 22:04:57.388680 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53dda91d-1cba-4026-8331-eccfee13427e None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:04:57.390269 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-53dda91d-1cba-4026-8331-eccfee13427e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:04:57.445847 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:04:57.506053 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53dda91d-1cba-4026-8331-eccfee13427e None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:04:57.593557 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e027a284-d3f8-4f70-a48d-7cf7e64c1811 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc77e1ea1ada4700b809beb1c980c5e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417e3e2f1333481795a089bef2e24bea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:04:57.594557 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e027a284-d3f8-4f70-a48d-7cf7e64c1811 service cinder] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:04:57.594846 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e027a284-d3f8-4f70-a48d-7cf7e64c1811 service cinder] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:04:57.595125 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e027a284-d3f8-4f70-a48d-7cf7e64c1811 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:04:57.597005 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-e027a284-d3f8-4f70-a48d-7cf7e64c1811 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:04:57.699687 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:04:57.714138 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e027a284-d3f8-4f70-a48d-7cf7e64c1811 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:04:57.715635 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e027a284-d3f8-4f70-a48d-7cf7e64c1811 service cinder] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:04:57.735656 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 690/1373] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jul 13 22:04:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 22:04:58.049711 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0ae949-397b-4055-a461-9b165e10657e None None] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:04:58.050167 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0ae949-397b-4055-a461-9b165e10657e None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:04:58.050514 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd0ae949-397b-4055-a461-9b165e10657e None None] PATH_INFO: `/` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:04:58.052669 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 684/1374] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jul 13 22:04:58 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 22:04:58.057963 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e0f261-3b04-4156-be21-1fa354e599fa None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:04:58.058253 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e0f261-3b04-4156-be21-1fa354e599fa None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:04:58.058558 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8e0f261-3b04-4156-be21-1fa354e599fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:04:58.126173 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-f8e0f261-3b04-4156-be21-1fa354e599fa None None] MFA Rules not processed for user `cc99c93da5074cf5a48bc2aa9a92c5e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 22:04:58.127731 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-f8e0f261-3b04-4156-be21-1fa354e599fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:04:58.232759 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 691/1375] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jul 13 22:04:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 22:04:58.277562 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0ae949-397b-4055-a461-9b165e10657e None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:04:58.279151 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-cd0ae949-397b-4055-a461-9b165e10657e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:04:58.434343 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd0ae949-397b-4055-a461-9b165e10657e None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:04:58.489105 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:04:58.556179 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eaf10694-28a3-4d89-8ad6-1df20be2da05 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84281091687464990ffbca590b9343b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417e3e2f1333481795a089bef2e24bea', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:04:58.556846 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf10694-28a3-4d89-8ad6-1df20be2da05 service nova] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:04:58.557129 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf10694-28a3-4d89-8ad6-1df20be2da05 service nova] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:04:58.557402 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eaf10694-28a3-4d89-8ad6-1df20be2da05 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:04:58.559106 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-eaf10694-28a3-4d89-8ad6-1df20be2da05 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:04:58.648736 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf10694-28a3-4d89-8ad6-1df20be2da05 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:04:58.650321 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eaf10694-28a3-4d89-8ad6-1df20be2da05 service nova] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:04:58.668382 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 685/1376] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jul 13 22:04:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 391 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 22:05:00.711388 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219f2dc0-cae6-40a6-a255-16a427d16206 None None] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:05:00.711911 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219f2dc0-cae6-40a6-a255-16a427d16206 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:05:00.711911 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219f2dc0-cae6-40a6-a255-16a427d16206 None None] PATH_INFO: `/` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:05:00.713540 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 692/1377] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jul 13 22:05:00 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 13 22:05:00.718760 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528614ed-0b4e-40ea-aa05-906bf798c484 None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:05:00.719212 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528614ed-0b4e-40ea-aa05-906bf798c484 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:05:00.719307 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-528614ed-0b4e-40ea-aa05-906bf798c484 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:05:00.806691 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-528614ed-0b4e-40ea-aa05-906bf798c484 None None] MFA Rules not processed for user `cc99c93da5074cf5a48bc2aa9a92c5e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 22:05:00.807824 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-528614ed-0b4e-40ea-aa05-906bf798c484 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:05:00.814753 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:05:00.911415 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 686/1378] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jul 13 22:05:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 13 22:05:00.958279 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219f2dc0-cae6-40a6-a255-16a427d16206 None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:05:00.959918 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-219f2dc0-cae6-40a6-a255-16a427d16206 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:05:01.028973 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:05:01.132837 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219f2dc0-cae6-40a6-a255-16a427d16206 None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:05:01.285756 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-451b9e1d-9194-4471-a22a-5cc1eaa6da2a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84281091687464990ffbca590b9343b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417e3e2f1333481795a089bef2e24bea', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:05:01.286708 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451b9e1d-9194-4471-a22a-5cc1eaa6da2a service nova] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:05:01.287088 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451b9e1d-9194-4471-a22a-5cc1eaa6da2a service nova] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:05:01.287631 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-451b9e1d-9194-4471-a22a-5cc1eaa6da2a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:05:01.290994 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-451b9e1d-9194-4471-a22a-5cc1eaa6da2a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:05:01.318342 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:05:01.393963 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451b9e1d-9194-4471-a22a-5cc1eaa6da2a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:05:01.395575 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-451b9e1d-9194-4471-a22a-5cc1eaa6da2a service nova] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:05:01.415352 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 693/1379] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jul 13 22:05:00 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 457 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 22:05:03.463427 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76280e92-9f62-43e4-b124-c38cb9692b06 None None] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:05:03.464043 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76280e92-9f62-43e4-b124-c38cb9692b06 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:05:03.464043 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76280e92-9f62-43e4-b124-c38cb9692b06 None None] PATH_INFO: `/` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:05:03.465579 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 687/1380] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jul 13 22:05:03 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 22:05:03.470641 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d28e9de-7557-493a-999d-23e6e06ba5ad None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:05:03.471091 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d28e9de-7557-493a-999d-23e6e06ba5ad None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:05:03.471174 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d28e9de-7557-493a-999d-23e6e06ba5ad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:05:03.542219 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-3d28e9de-7557-493a-999d-23e6e06ba5ad None None] MFA Rules not processed for user `cc99c93da5074cf5a48bc2aa9a92c5e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 22:05:03.543460 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-3d28e9de-7557-493a-999d-23e6e06ba5ad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:05:03.650329 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:05:03.658646 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 694/1381] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jul 13 22:05:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 22:05:03.701729 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76280e92-9f62-43e4-b124-c38cb9692b06 None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:05:03.703313 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-76280e92-9f62-43e4-b124-c38cb9692b06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:05:03.840908 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:05:03.851676 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76280e92-9f62-43e4-b124-c38cb9692b06 None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:05:03.969030 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e54336ba-33ba-4728-977f-59d020c34ff3 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84281091687464990ffbca590b9343b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417e3e2f1333481795a089bef2e24bea', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:05:03.969689 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54336ba-33ba-4728-977f-59d020c34ff3 service nova] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:05:03.969962 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54336ba-33ba-4728-977f-59d020c34ff3 service nova] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:05:03.970369 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e54336ba-33ba-4728-977f-59d020c34ff3 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:05:03.972112 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-e54336ba-33ba-4728-977f-59d020c34ff3 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:05:04.075800 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54336ba-33ba-4728-977f-59d020c34ff3 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:05:04.077738 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e54336ba-33ba-4728-977f-59d020c34ff3 service nova] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:05:04.091044 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:05:04.104624 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 688/1382] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jul 13 22:05:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 403 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 22:05:06.163077 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695586ca-abce-4c4f-92d8-2070eb847c3f None None] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:05:06.163077 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695586ca-abce-4c4f-92d8-2070eb847c3f None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:05:06.163893 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-695586ca-abce-4c4f-92d8-2070eb847c3f None None] PATH_INFO: `/` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:05:06.164612 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 695/1383] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jul 13 22:05:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 22:05:06.171180 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86dcd52c-2369-475c-a390-8a00d269dc3e None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:05:06.171652 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86dcd52c-2369-475c-a390-8a00d269dc3e None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:05:06.171760 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-86dcd52c-2369-475c-a390-8a00d269dc3e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:05:06.250121 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-86dcd52c-2369-475c-a390-8a00d269dc3e None None] MFA Rules not processed for user `cc99c93da5074cf5a48bc2aa9a92c5e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 22:05:06.251393 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-86dcd52c-2369-475c-a390-8a00d269dc3e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:05:06.352345 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 689/1384] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jul 13 22:05:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 182 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 22:05:06.403873 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695586ca-abce-4c4f-92d8-2070eb847c3f None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:05:06.405506 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-695586ca-abce-4c4f-92d8-2070eb847c3f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:05:06.588304 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-695586ca-abce-4c4f-92d8-2070eb847c3f None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:05:06.674501 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:05:06.723072 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33b7ad6f-ef90-467b-8cb6-712bb5322a6d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84281091687464990ffbca590b9343b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417e3e2f1333481795a089bef2e24bea', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:05:06.723741 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b7ad6f-ef90-467b-8cb6-712bb5322a6d service nova] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:05:06.724022 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b7ad6f-ef90-467b-8cb6-712bb5322a6d service nova] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:05:06.724306 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33b7ad6f-ef90-467b-8cb6-712bb5322a6d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:05:06.726004 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-33b7ad6f-ef90-467b-8cb6-712bb5322a6d service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:05:06.824859 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b7ad6f-ef90-467b-8cb6-712bb5322a6d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:05:06.826429 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33b7ad6f-ef90-467b-8cb6-712bb5322a6d service nova] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:05:06.846574 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 696/1385] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jul 13 22:05:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 443 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 13 22:05:09.334209 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199293fe-bd5b-40c8-80eb-50e7fe00fc6a None None] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:05:09.334833 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199293fe-bd5b-40c8-80eb-50e7fe00fc6a None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:05:09.334929 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-199293fe-bd5b-40c8-80eb-50e7fe00fc6a None None] PATH_INFO: `/` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:05:09.336708 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 690/1386] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jul 13 22:05:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 22:05:09.343148 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d9b3bc-c1ed-4468-8ff1-bb421f493ebf None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:05:09.343716 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d9b3bc-c1ed-4468-8ff1-bb421f493ebf None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:05:09.343716 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90d9b3bc-c1ed-4468-8ff1-bb421f493ebf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:05:09.413524 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-90d9b3bc-c1ed-4468-8ff1-bb421f493ebf None None] MFA Rules not processed for user `cc99c93da5074cf5a48bc2aa9a92c5e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 22:05:09.414857 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-90d9b3bc-c1ed-4468-8ff1-bb421f493ebf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:05:09.449838 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:05:09.518579 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 697/1387] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jul 13 22:05:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 22:05:09.564340 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199293fe-bd5b-40c8-80eb-50e7fe00fc6a None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:05:09.566785 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-199293fe-bd5b-40c8-80eb-50e7fe00fc6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:05:09.669579 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:05:09.781758 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-199293fe-bd5b-40c8-80eb-50e7fe00fc6a None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:05:09.922183 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e66e4c10-26f3-470d-a9bb-dc7014ad4358 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84281091687464990ffbca590b9343b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417e3e2f1333481795a089bef2e24bea', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:05:09.923078 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66e4c10-26f3-470d-a9bb-dc7014ad4358 service nova] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:05:09.923443 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66e4c10-26f3-470d-a9bb-dc7014ad4358 service nova] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:05:09.923775 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e66e4c10-26f3-470d-a9bb-dc7014ad4358 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:05:09.925876 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-e66e4c10-26f3-470d-a9bb-dc7014ad4358 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:05:09.967794 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:05:10.025015 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66e4c10-26f3-470d-a9bb-dc7014ad4358 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:05:10.026709 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e66e4c10-26f3-470d-a9bb-dc7014ad4358 service nova] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:05:10.046731 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 691/1388] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jul 13 22:05:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 483 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 22:05:12.157765 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37cf27b-5522-4a7d-93e3-e1d405412f8f None None] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:05:12.158332 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37cf27b-5522-4a7d-93e3-e1d405412f8f None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:05:12.158332 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f37cf27b-5522-4a7d-93e3-e1d405412f8f None None] PATH_INFO: `/` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:05:12.159763 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 698/1389] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jul 13 22:05:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 22:05:12.165211 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8728a08f-a8f9-420e-9079-397599820af5 None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:05:12.165667 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8728a08f-a8f9-420e-9079-397599820af5 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:05:12.165797 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8728a08f-a8f9-420e-9079-397599820af5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:05:12.232865 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-8728a08f-a8f9-420e-9079-397599820af5 None None] MFA Rules not processed for user `cc99c93da5074cf5a48bc2aa9a92c5e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 22:05:12.234062 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-8728a08f-a8f9-420e-9079-397599820af5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:05:12.340039 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 692/1390] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jul 13 22:05:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 22:05:12.384382 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37cf27b-5522-4a7d-93e3-e1d405412f8f None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:05:12.386048 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-f37cf27b-5522-4a7d-93e3-e1d405412f8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:05:12.541992 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f37cf27b-5522-4a7d-93e3-e1d405412f8f None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:05:12.559267 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:05:12.659916 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f658ebbc-ba86-4a9d-b9c6-a78ae7b81201 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84281091687464990ffbca590b9343b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417e3e2f1333481795a089bef2e24bea', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:05:12.660578 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f658ebbc-ba86-4a9d-b9c6-a78ae7b81201 service nova] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:05:12.660858 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f658ebbc-ba86-4a9d-b9c6-a78ae7b81201 service nova] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:05:12.661128 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f658ebbc-ba86-4a9d-b9c6-a78ae7b81201 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:05:12.662808 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-f658ebbc-ba86-4a9d-b9c6-a78ae7b81201 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:05:12.733435 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da00afda-27db-49bc-ade6-2cf8fe5e7479 None None] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:05:12.733804 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da00afda-27db-49bc-ade6-2cf8fe5e7479 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:05:12.734452 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da00afda-27db-49bc-ade6-2cf8fe5e7479 None None] PATH_INFO: `/` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:05:12.736408 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 693/1391] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jul 13 22:05:12 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 13 22:05:12.744905 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4f75fa-9295-4954-a49c-fdf74a4878f8 None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:05:12.745724 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4f75fa-9295-4954-a49c-fdf74a4878f8 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:05:12.745724 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6f4f75fa-9295-4954-a49c-fdf74a4878f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:05:12.765933 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:05:12.780214 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f658ebbc-ba86-4a9d-b9c6-a78ae7b81201 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:05:12.782342 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f658ebbc-ba86-4a9d-b9c6-a78ae7b81201 service nova] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:05:12.804611 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 699/1392] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jul 13 22:05:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 421 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 22:05:12.829714 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-6f4f75fa-9295-4954-a49c-fdf74a4878f8 None None] MFA Rules not processed for user `cc99c93da5074cf5a48bc2aa9a92c5e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 22:05:12.831282 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-6f4f75fa-9295-4954-a49c-fdf74a4878f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:05:12.937960 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 694/1393] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jul 13 22:05:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 22:05:12.997106 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442068dd-5586-4475-83b7-5703117e5b1b None None] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:05:12.997383 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442068dd-5586-4475-83b7-5703117e5b1b None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:05:12.997638 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442068dd-5586-4475-83b7-5703117e5b1b None None] PATH_INFO: `/` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:05:12.999107 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 700/1394] 10.222.0.52 () {54 vars in 962 bytes} [Wed Jul 13 22:05:12 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 22:05:13.004793 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389f12c3-60d1-450d-8499-60e3b05b3011 None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:05:13.005069 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389f12c3-60d1-450d-8499-60e3b05b3011 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:05:13.005351 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-389f12c3-60d1-450d-8499-60e3b05b3011 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:05:13.083954 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-389f12c3-60d1-450d-8499-60e3b05b3011 None None] MFA Rules not processed for user `e4d35d50e15d43d3afc59206b8165f30`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 22:05:13.085049 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-389f12c3-60d1-450d-8499-60e3b05b3011 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:05:13.122198 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:05:13.168850 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 695/1395] 10.222.0.52 () {58 vars in 1059 bytes} [Wed Jul 13 22:05:13 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 165 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 22:05:13.177120 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442068dd-5586-4475-83b7-5703117e5b1b None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:05:13.178692 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-442068dd-5586-4475-83b7-5703117e5b1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:05:13.203200 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:05:13.287077 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442068dd-5586-4475-83b7-5703117e5b1b None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:05:13.362216 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa5f723d-16cf-49c0-8801-b3872b6c0d44 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4d35d50e15d43d3afc59206b8165f30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417e3e2f1333481795a089bef2e24bea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:05:13.362808 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5f723d-16cf-49c0-8801-b3872b6c0d44 service glance] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:05:13.363088 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5f723d-16cf-49c0-8801-b3872b6c0d44 service glance] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:05:13.363369 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5f723d-16cf-49c0-8801-b3872b6c0d44 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:05:13.365007 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-aa5f723d-16cf-49c0-8801-b3872b6c0d44 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:05:13.441468 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:05:13.480483 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa5f723d-16cf-49c0-8801-b3872b6c0d44 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:05:13.481942 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa5f723d-16cf-49c0-8801-b3872b6c0d44 service glance] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:05:13.502212 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 701/1396] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jul 13 22:05:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 325 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 22:05:14.230053 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be260b45-b55a-4376-b8de-ae8dc74adfcc None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:05:14.230562 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be260b45-b55a-4376-b8de-ae8dc74adfcc None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:05:14.230682 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be260b45-b55a-4376-b8de-ae8dc74adfcc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:05:14.296193 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-be260b45-b55a-4376-b8de-ae8dc74adfcc None None] MFA Rules not processed for user `b866bacdafae4db6a32714c07c56515f`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 22:05:14.297418 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-be260b45-b55a-4376-b8de-ae8dc74adfcc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:05:14.391857 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 696/1397] 10.222.0.52 () {58 vars in 1009 bytes} [Wed Jul 13 22:05:14 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 22:05:14.438137 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b531ae-671d-45b3-9613-2ef4950297ee None None] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:05:14.438678 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b531ae-671d-45b3-9613-2ef4950297ee None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:05:14.438678 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87b531ae-671d-45b3-9613-2ef4950297ee None None] PATH_INFO: `/` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:05:14.440307 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 702/1398] 10.222.0.52 () {54 vars in 942 bytes} [Wed Jul 13 22:05:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 22:05:14.447035 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ef46af-72a4-4028-97c8-24ef65c9b1fb None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:05:14.447325 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ef46af-72a4-4028-97c8-24ef65c9b1fb None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:05:14.447604 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7ef46af-72a4-4028-97c8-24ef65c9b1fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:05:14.506974 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:05:14.526729 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-a7ef46af-72a4-4028-97c8-24ef65c9b1fb None None] MFA Rules not processed for user `1eb1525233ba48c99b15d4cb75db4822`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 22:05:14.527933 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-a7ef46af-72a4-4028-97c8-24ef65c9b1fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:05:14.648302 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 697/1399] 10.222.0.52 () {58 vars in 1039 bytes} [Wed Jul 13 22:05:14 2022] POST /identity/v3/auth/tokens => generated 3294 bytes in 203 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 13 22:05:14.657884 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b531ae-671d-45b3-9613-2ef4950297ee None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:05:14.659485 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-87b531ae-671d-45b3-9613-2ef4950297ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:05:14.796594 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87b531ae-671d-45b3-9613-2ef4950297ee None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:05:14.838477 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:05:14.871118 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3876c7-2f4f-4074-9f49-14da516a9c45 None None] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:05:14.871408 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3876c7-2f4f-4074-9f49-14da516a9c45 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:05:14.871682 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3876c7-2f4f-4074-9f49-14da516a9c45 None None] PATH_INFO: `/` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:05:14.873076 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 698/1400] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jul 13 22:05:14 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 22:05:14.878216 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8740cd1-1b34-4ca7-b9b0-be89d2a624fa None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:05:14.878513 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8740cd1-1b34-4ca7-b9b0-be89d2a624fa None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:05:14.878823 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c8740cd1-1b34-4ca7-b9b0-be89d2a624fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:05:14.904145 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7173c0-3f9e-4145-8157-a4df27d6962a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eb1525233ba48c99b15d4cb75db4822', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417e3e2f1333481795a089bef2e24bea', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:05:14.904767 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7173c0-3f9e-4145-8157-a4df27d6962a service swift] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:05:14.905042 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7173c0-3f9e-4145-8157-a4df27d6962a service swift] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:05:14.905314 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca7173c0-3f9e-4145-8157-a4df27d6962a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:05:14.907027 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-ca7173c0-3f9e-4145-8157-a4df27d6962a service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:05:14.946063 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-c8740cd1-1b34-4ca7-b9b0-be89d2a624fa None None] MFA Rules not processed for user `cc99c93da5074cf5a48bc2aa9a92c5e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 22:05:14.947336 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-c8740cd1-1b34-4ca7-b9b0-be89d2a624fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:05:14.994219 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7173c0-3f9e-4145-8157-a4df27d6962a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:05:14.995948 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca7173c0-3f9e-4145-8157-a4df27d6962a service swift] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:05:15.003929 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:05:15.004156 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 703/1401] 10.222.0.52 () {60 vars in 1384 bytes} [Wed Jul 13 22:05:14 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 13 22:05:15.047459 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 699/1402] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jul 13 22:05:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 13 22:05:15.093559 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7173c0-3f9e-4145-8157-a4df27d6962a service swift] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:05:15.094812 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-ca7173c0-3f9e-4145-8157-a4df27d6962a service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:05:15.180199 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:05:15.205355 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca7173c0-3f9e-4145-8157-a4df27d6962a service swift] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:05:15.330995 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00313e08-c115-407f-9d2d-7199b478f9af service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84281091687464990ffbca590b9343b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417e3e2f1333481795a089bef2e24bea', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:05:15.331591 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00313e08-c115-407f-9d2d-7199b478f9af service nova] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:05:15.331877 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00313e08-c115-407f-9d2d-7199b478f9af service nova] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:05:15.332158 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00313e08-c115-407f-9d2d-7199b478f9af service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:05:15.333826 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-00313e08-c115-407f-9d2d-7199b478f9af service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:05:15.434350 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00313e08-c115-407f-9d2d-7199b478f9af service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:05:15.436142 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00313e08-c115-407f-9d2d-7199b478f9af service nova] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:05:15.450978 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:05:15.459465 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 704/1403] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jul 13 22:05:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 366 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 22:05:17.580676 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35484ece-a40c-4057-bbe9-cb3aff7f83d9 None None] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:05:17.581279 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35484ece-a40c-4057-bbe9-cb3aff7f83d9 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:05:17.581279 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35484ece-a40c-4057-bbe9-cb3aff7f83d9 None None] PATH_INFO: `/` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:05:17.582656 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 700/1404] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jul 13 22:05:17 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 22:05:17.588177 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05ccdb7-b717-467e-b28f-e5d7dbacb9e1 None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:05:17.588635 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05ccdb7-b717-467e-b28f-e5d7dbacb9e1 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:05:17.588728 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a05ccdb7-b717-467e-b28f-e5d7dbacb9e1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:05:17.659632 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-a05ccdb7-b717-467e-b28f-e5d7dbacb9e1 None None] MFA Rules not processed for user `cc99c93da5074cf5a48bc2aa9a92c5e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 22:05:17.660846 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-a05ccdb7-b717-467e-b28f-e5d7dbacb9e1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:05:17.778641 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 705/1405] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jul 13 22:05:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 22:05:17.815134 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35484ece-a40c-4057-bbe9-cb3aff7f83d9 None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:05:17.816650 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-35484ece-a40c-4057-bbe9-cb3aff7f83d9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:05:17.974476 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35484ece-a40c-4057-bbe9-cb3aff7f83d9 None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:05:18.019704 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:05:18.098401 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eb00bafb-42bf-43f6-8ffa-a813c7cfa664 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84281091687464990ffbca590b9343b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417e3e2f1333481795a089bef2e24bea', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:05:18.098991 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb00bafb-42bf-43f6-8ffa-a813c7cfa664 service nova] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:05:18.099268 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb00bafb-42bf-43f6-8ffa-a813c7cfa664 service nova] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:05:18.099540 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eb00bafb-42bf-43f6-8ffa-a813c7cfa664 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:05:18.101305 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-eb00bafb-42bf-43f6-8ffa-a813c7cfa664 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:05:18.205329 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb00bafb-42bf-43f6-8ffa-a813c7cfa664 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:05:18.206977 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eb00bafb-42bf-43f6-8ffa-a813c7cfa664 service nova] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:05:18.226928 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 701/1406] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jul 13 22:05:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 412 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 22:05:20.297744 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb8b637-a286-4786-bafc-fae3841626ac None None] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:05:20.298334 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb8b637-a286-4786-bafc-fae3841626ac None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:05:20.298334 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcb8b637-a286-4786-bafc-fae3841626ac None None] PATH_INFO: `/` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:05:20.299910 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 706/1407] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jul 13 22:05:20 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 22:05:20.306545 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb596db4-3608-49ac-ba3c-5d60adf359fc None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:05:20.306545 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb596db4-3608-49ac-ba3c-5d60adf359fc None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:05:20.306545 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb596db4-3608-49ac-ba3c-5d60adf359fc None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:05:20.374132 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:05:20.383997 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-bb596db4-3608-49ac-ba3c-5d60adf359fc None None] MFA Rules not processed for user `cc99c93da5074cf5a48bc2aa9a92c5e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 22:05:20.385255 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-bb596db4-3608-49ac-ba3c-5d60adf359fc None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:05:20.495998 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 702/1408] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jul 13 22:05:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 22:05:20.542030 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb8b637-a286-4786-bafc-fae3841626ac None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:05:20.543606 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-dcb8b637-a286-4786-bafc-fae3841626ac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:05:20.643641 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:05:20.722765 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcb8b637-a286-4786-bafc-fae3841626ac None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:05:20.841841 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4168d19-1a84-4b74-ada6-98ad049f9cf4 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84281091687464990ffbca590b9343b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417e3e2f1333481795a089bef2e24bea', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:05:20.842569 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4168d19-1a84-4b74-ada6-98ad049f9cf4 service nova] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:05:20.842856 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4168d19-1a84-4b74-ada6-98ad049f9cf4 service nova] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:05:20.843143 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4168d19-1a84-4b74-ada6-98ad049f9cf4 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:05:20.844823 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-b4168d19-1a84-4b74-ada6-98ad049f9cf4 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:05:20.882119 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:05:20.946208 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4168d19-1a84-4b74-ada6-98ad049f9cf4 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:05:20.947870 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4168d19-1a84-4b74-ada6-98ad049f9cf4 service nova] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:05:20.984425 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 707/1409] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jul 13 22:05:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 443 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 22:05:30.493868 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=44 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:05:30.968410 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=31 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:05:37.134466 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca93fff-7f88-4961-9786-83f73270fffc None None] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:05:37.135096 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca93fff-7f88-4961-9786-83f73270fffc None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:05:37.135096 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ca93fff-7f88-4961-9786-83f73270fffc None None] PATH_INFO: `/` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:05:37.136481 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 703/1410] 10.222.0.52 () {54 vars in 918 bytes} [Wed Jul 13 22:05:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 22:05:37.141866 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0d630d-4ad2-4e69-bacd-ee16ce644528 None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:05:37.142323 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0d630d-4ad2-4e69-bacd-ee16ce644528 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:05:37.142432 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d0d630d-4ad2-4e69-bacd-ee16ce644528 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:05:37.226645 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-9d0d630d-4ad2-4e69-bacd-ee16ce644528 None None] MFA Rules not processed for user `ac673d641d8849c79bf4cf71c9805587`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 22:05:37.227946 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-9d0d630d-4ad2-4e69-bacd-ee16ce644528 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:05:37.347051 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 708/1411] 10.222.0.52 () {58 vars in 1015 bytes} [Wed Jul 13 22:05:37 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 206 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 22:05:37.362022 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d9194c-d6b1-43c6-a603-23df754e9353 None None] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:05:37.362309 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d9194c-d6b1-43c6-a603-23df754e9353 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:05:37.362613 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6d9194c-d6b1-43c6-a603-23df754e9353 None None] PATH_INFO: `/` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:05:37.364025 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 704/1412] 10.222.0.52 () {54 vars in 918 bytes} [Wed Jul 13 22:05:37 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 22:05:37.369093 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e32d37-6495-45fd-b4b4-2e5cbf60c2a7 None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:05:37.369384 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e32d37-6495-45fd-b4b4-2e5cbf60c2a7 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:05:37.369672 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e32d37-6495-45fd-b4b4-2e5cbf60c2a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:05:37.448852 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:05:37.451941 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-28e32d37-6495-45fd-b4b4-2e5cbf60c2a7 None None] MFA Rules not processed for user `f84281091687464990ffbca590b9343b`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 22:05:37.453973 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-28e32d37-6495-45fd-b4b4-2e5cbf60c2a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:05:37.581692 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 709/1413] 10.222.0.52 () {58 vars in 1015 bytes} [Wed Jul 13 22:05:37 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 214 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 13 22:05:40.145683 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857d5cf7-a88f-4815-8b45-ee29ede3601a None None] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:05:40.146241 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857d5cf7-a88f-4815-8b45-ee29ede3601a None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:05:40.146241 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-857d5cf7-a88f-4815-8b45-ee29ede3601a None None] PATH_INFO: `/` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:05:40.147604 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 705/1414] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 13 22:05:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 22:05:40.154588 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830ab0cd-6283-49e7-a5f2-ddc9a2a66931 None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:05:40.155038 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830ab0cd-6283-49e7-a5f2-ddc9a2a66931 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:05:40.155151 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-830ab0cd-6283-49e7-a5f2-ddc9a2a66931 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:05:40.222279 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-830ab0cd-6283-49e7-a5f2-ddc9a2a66931 None None] MFA Rules not processed for user `cc99c93da5074cf5a48bc2aa9a92c5e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 22:05:40.223628 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-830ab0cd-6283-49e7-a5f2-ddc9a2a66931 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:05:40.279120 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:05:40.322549 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 710/1415] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 13 22:05:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 22:05:40.365368 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857d5cf7-a88f-4815-8b45-ee29ede3601a None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:05:40.366951 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-857d5cf7-a88f-4815-8b45-ee29ede3601a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:05:40.522337 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-857d5cf7-a88f-4815-8b45-ee29ede3601a None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:05:40.609761 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:05:40.652369 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb42e1f-2d76-49e8-bb1e-9b355d794c58 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84281091687464990ffbca590b9343b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417e3e2f1333481795a089bef2e24bea', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:05:40.653022 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb42e1f-2d76-49e8-bb1e-9b355d794c58 service nova] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:05:40.653300 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb42e1f-2d76-49e8-bb1e-9b355d794c58 service nova] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:05:40.653570 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb42e1f-2d76-49e8-bb1e-9b355d794c58 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:05:40.655310 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-1cb42e1f-2d76-49e8-bb1e-9b355d794c58 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:05:40.755206 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb42e1f-2d76-49e8-bb1e-9b355d794c58 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:05:40.757616 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1cb42e1f-2d76-49e8-bb1e-9b355d794c58 service nova] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:05:40.776520 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 706/1416] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jul 13 22:05:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 412 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 13 22:05:41.677803 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177c7dbd-51d0-48f8-a188-a0b143acff89 None None] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:05:41.678774 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177c7dbd-51d0-48f8-a188-a0b143acff89 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:05:41.678774 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-177c7dbd-51d0-48f8-a188-a0b143acff89 None None] PATH_INFO: `/` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:05:41.681070 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 711/1417] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jul 13 22:05:41 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 22:05:41.687010 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d3600b-3838-4166-89bc-6395d33a8a63 None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:05:41.687487 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d3600b-3838-4166-89bc-6395d33a8a63 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:05:41.687600 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d3600b-3838-4166-89bc-6395d33a8a63 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:05:41.776479 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-e0d3600b-3838-4166-89bc-6395d33a8a63 None None] MFA Rules not processed for user `cc99c93da5074cf5a48bc2aa9a92c5e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 22:05:41.777678 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-e0d3600b-3838-4166-89bc-6395d33a8a63 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:05:41.805130 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:05:41.877806 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 707/1418] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jul 13 22:05:41 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 22:05:42.031485 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177c7dbd-51d0-48f8-a188-a0b143acff89 None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:05:42.033050 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-177c7dbd-51d0-48f8-a188-a0b143acff89 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:05:42.189403 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-177c7dbd-51d0-48f8-a188-a0b143acff89 None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:05:42.230828 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:05:42.310445 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5751089b-711c-430d-9f43-c6dc44e865ba service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84281091687464990ffbca590b9343b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417e3e2f1333481795a089bef2e24bea', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:05:42.311134 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5751089b-711c-430d-9f43-c6dc44e865ba service nova] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:05:42.311417 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5751089b-711c-430d-9f43-c6dc44e865ba service nova] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:05:42.311692 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5751089b-711c-430d-9f43-c6dc44e865ba service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:05:42.313366 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-5751089b-711c-430d-9f43-c6dc44e865ba service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:05:42.411757 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5751089b-711c-430d-9f43-c6dc44e865ba service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:05:42.413396 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5751089b-711c-430d-9f43-c6dc44e865ba service nova] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:05:42.433133 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 712/1419] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jul 13 22:05:42 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 402 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 22:05:43.332554 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e059f8-68ca-4639-a5cd-f5b4295f5042 None None] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:05:43.333142 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e059f8-68ca-4639-a5cd-f5b4295f5042 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:05:43.333142 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5e059f8-68ca-4639-a5cd-f5b4295f5042 None None] PATH_INFO: `/` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:05:43.334557 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 708/1420] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jul 13 22:05:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 22:05:43.340903 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa49b0b7-202b-4915-bba9-56442a857f54 None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:05:43.341653 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa49b0b7-202b-4915-bba9-56442a857f54 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:05:43.341795 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa49b0b7-202b-4915-bba9-56442a857f54 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:05:43.409614 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:05:43.433097 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-fa49b0b7-202b-4915-bba9-56442a857f54 None None] MFA Rules not processed for user `cc99c93da5074cf5a48bc2aa9a92c5e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 22:05:43.434324 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-fa49b0b7-202b-4915-bba9-56442a857f54 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:05:43.540273 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 713/1421] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jul 13 22:05:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 22:05:43.702391 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e059f8-68ca-4639-a5cd-f5b4295f5042 None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:05:43.704618 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-f5e059f8-68ca-4639-a5cd-f5b4295f5042 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:05:43.863889 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5e059f8-68ca-4639-a5cd-f5b4295f5042 None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:05:43.876222 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:05:43.982593 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28cd97ba-1c67-49da-8550-607cf712f106 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84281091687464990ffbca590b9343b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417e3e2f1333481795a089bef2e24bea', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:05:43.983161 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cd97ba-1c67-49da-8550-607cf712f106 service nova] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:05:43.983435 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cd97ba-1c67-49da-8550-607cf712f106 service nova] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:05:43.983703 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28cd97ba-1c67-49da-8550-607cf712f106 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:05:43.985364 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-28cd97ba-1c67-49da-8550-607cf712f106 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:05:44.080330 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28cd97ba-1c67-49da-8550-607cf712f106 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:05:44.081927 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28cd97ba-1c67-49da-8550-607cf712f106 service nova] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:05:44.100492 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 709/1422] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jul 13 22:05:43 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 399 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 22:05:44.966402 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d178fe-cc4f-431a-81d8-46511631b6b1 None None] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:05:44.967198 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d178fe-cc4f-431a-81d8-46511631b6b1 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:05:44.967198 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d178fe-cc4f-431a-81d8-46511631b6b1 None None] PATH_INFO: `/` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:05:44.968398 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 714/1423] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jul 13 22:05:44 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 22:05:44.974717 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada0510f-d72e-4a60-b4c3-3d34b030563f None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:05:44.975415 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada0510f-d72e-4a60-b4c3-3d34b030563f None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:05:44.975565 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ada0510f-d72e-4a60-b4c3-3d34b030563f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:05:44.996029 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:05:45.058123 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-ada0510f-d72e-4a60-b4c3-3d34b030563f None None] MFA Rules not processed for user `cc99c93da5074cf5a48bc2aa9a92c5e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 22:05:45.059390 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-ada0510f-d72e-4a60-b4c3-3d34b030563f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:05:45.163383 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 710/1424] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jul 13 22:05:44 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 191 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 22:05:45.313572 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d178fe-cc4f-431a-81d8-46511631b6b1 None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:05:45.315028 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-76d178fe-cc4f-431a-81d8-46511631b6b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:05:45.430141 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:05:45.467368 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d178fe-cc4f-431a-81d8-46511631b6b1 None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:05:45.589761 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8674568c-28a6-4071-837a-93130e6864e9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84281091687464990ffbca590b9343b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417e3e2f1333481795a089bef2e24bea', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:05:45.590318 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8674568c-28a6-4071-837a-93130e6864e9 service nova] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:05:45.590633 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8674568c-28a6-4071-837a-93130e6864e9 service nova] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:05:45.590892 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8674568c-28a6-4071-837a-93130e6864e9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:05:45.592554 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-8674568c-28a6-4071-837a-93130e6864e9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:05:45.687647 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:05:45.703420 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8674568c-28a6-4071-837a-93130e6864e9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:05:45.705059 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8674568c-28a6-4071-837a-93130e6864e9 service nova] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:05:45.724875 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 715/1425] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jul 13 22:05:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 412 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 22:05:46.587365 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1353766d-e209-47bf-ac54-1f26ee3caba3 None None] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:05:46.587880 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1353766d-e209-47bf-ac54-1f26ee3caba3 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:05:46.587880 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1353766d-e209-47bf-ac54-1f26ee3caba3 None None] PATH_INFO: `/` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:05:46.589349 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 711/1426] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jul 13 22:05:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 22:05:46.594390 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11483859-90a7-45f5-ae59-78f4692c07c9 None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:05:46.594858 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11483859-90a7-45f5-ae59-78f4692c07c9 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:05:46.594967 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11483859-90a7-45f5-ae59-78f4692c07c9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:05:46.678754 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-11483859-90a7-45f5-ae59-78f4692c07c9 None None] MFA Rules not processed for user `cc99c93da5074cf5a48bc2aa9a92c5e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 22:05:46.679990 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-11483859-90a7-45f5-ae59-78f4692c07c9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:05:46.785179 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 716/1427] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jul 13 22:05:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 192 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 22:05:46.944897 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1353766d-e209-47bf-ac54-1f26ee3caba3 None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:05:46.947477 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-1353766d-e209-47bf-ac54-1f26ee3caba3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:05:47.051168 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:05:47.126591 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1353766d-e209-47bf-ac54-1f26ee3caba3 None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:05:47.263606 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-249ec384-829f-494c-b2cb-31476b890cab service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84281091687464990ffbca590b9343b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417e3e2f1333481795a089bef2e24bea', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:05:47.264264 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249ec384-829f-494c-b2cb-31476b890cab service nova] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:05:47.264548 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249ec384-829f-494c-b2cb-31476b890cab service nova] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:05:47.264806 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-249ec384-829f-494c-b2cb-31476b890cab service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:05:47.266509 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-249ec384-829f-494c-b2cb-31476b890cab service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:05:47.316973 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:05:47.359475 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249ec384-829f-494c-b2cb-31476b890cab service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:05:47.361086 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-249ec384-829f-494c-b2cb-31476b890cab service nova] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:05:47.379712 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 712/1428] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jul 13 22:05:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 436 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 22:05:48.184223 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ed08fc-630a-4223-97bd-f1b180bdaf2c None None] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:05:48.184904 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ed08fc-630a-4223-97bd-f1b180bdaf2c None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:05:48.184904 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73ed08fc-630a-4223-97bd-f1b180bdaf2c None None] PATH_INFO: `/` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:05:48.186144 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 717/1429] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jul 13 22:05:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 22:05:48.192135 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447a6fac-549b-4bcd-bd2b-e6f576b45157 None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:05:48.192625 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447a6fac-549b-4bcd-bd2b-e6f576b45157 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:05:48.192734 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-447a6fac-549b-4bcd-bd2b-e6f576b45157 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:05:48.281298 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-447a6fac-549b-4bcd-bd2b-e6f576b45157 None None] MFA Rules not processed for user `cc99c93da5074cf5a48bc2aa9a92c5e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 22:05:48.282514 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-447a6fac-549b-4bcd-bd2b-e6f576b45157 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:05:48.383383 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 713/1430] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jul 13 22:05:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 193 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 22:05:48.533559 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ed08fc-630a-4223-97bd-f1b180bdaf2c None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:05:48.535020 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-73ed08fc-630a-4223-97bd-f1b180bdaf2c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:05:48.589100 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:05:48.699955 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73ed08fc-630a-4223-97bd-f1b180bdaf2c None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:05:48.834871 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d195bdee-e824-44ed-a105-2e485e084e7a service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84281091687464990ffbca590b9343b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417e3e2f1333481795a089bef2e24bea', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:05:48.835447 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d195bdee-e824-44ed-a105-2e485e084e7a service nova] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:05:48.835730 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d195bdee-e824-44ed-a105-2e485e084e7a service nova] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:05:48.836047 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d195bdee-e824-44ed-a105-2e485e084e7a service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:05:48.837738 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-d195bdee-e824-44ed-a105-2e485e084e7a service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:05:48.845020 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:05:48.931263 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d195bdee-e824-44ed-a105-2e485e084e7a service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:05:48.932864 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d195bdee-e824-44ed-a105-2e485e084e7a service nova] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:05:48.952403 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 718/1431] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jul 13 22:05:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 419 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 22:05:49.749096 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79ed346-0daf-44c5-9412-f0756cb08120 None None] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:05:49.749709 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79ed346-0daf-44c5-9412-f0756cb08120 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:05:49.749709 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c79ed346-0daf-44c5-9412-f0756cb08120 None None] PATH_INFO: `/` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:05:49.751092 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 714/1432] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jul 13 22:05:49 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 22:05:49.757172 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cd5fd2-4ba8-444b-bfca-6eee47e8ba75 None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:05:49.757932 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cd5fd2-4ba8-444b-bfca-6eee47e8ba75 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:05:49.758397 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5cd5fd2-4ba8-444b-bfca-6eee47e8ba75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:05:49.839049 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-f5cd5fd2-4ba8-444b-bfca-6eee47e8ba75 None None] MFA Rules not processed for user `cc99c93da5074cf5a48bc2aa9a92c5e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 22:05:49.840185 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-f5cd5fd2-4ba8-444b-bfca-6eee47e8ba75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:05:49.912548 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:05:49.943169 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 719/1433] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jul 13 22:05:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 188 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 22:05:50.099976 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79ed346-0daf-44c5-9412-f0756cb08120 None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:05:50.101436 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-c79ed346-0daf-44c5-9412-f0756cb08120 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:05:50.120539 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:05:50.258917 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c79ed346-0daf-44c5-9412-f0756cb08120 None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:05:50.350883 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:05:50.378355 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf7a1cf7-a7eb-415e-9919-099c5b7053f0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84281091687464990ffbca590b9343b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417e3e2f1333481795a089bef2e24bea', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:05:50.379037 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7a1cf7-a7eb-415e-9919-099c5b7053f0 service nova] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:05:50.379317 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7a1cf7-a7eb-415e-9919-099c5b7053f0 service nova] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:05:50.379589 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf7a1cf7-a7eb-415e-9919-099c5b7053f0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:05:50.381251 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-cf7a1cf7-a7eb-415e-9919-099c5b7053f0 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:05:50.479962 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7a1cf7-a7eb-415e-9919-099c5b7053f0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:05:50.481601 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf7a1cf7-a7eb-415e-9919-099c5b7053f0 service nova] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:05:50.505313 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 715/1434] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jul 13 22:05:50 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 406 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 22:05:51.304163 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba9bfce-3193-47c6-b530-b417a5c8f570 None None] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:05:51.304723 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba9bfce-3193-47c6-b530-b417a5c8f570 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:05:51.304723 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba9bfce-3193-47c6-b530-b417a5c8f570 None None] PATH_INFO: `/` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:05:51.306032 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 720/1435] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jul 13 22:05:51 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 22:05:51.311124 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a7b1e7-98cf-4d6d-9798-2a3b535d9eaa None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:05:51.311599 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a7b1e7-98cf-4d6d-9798-2a3b535d9eaa None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:05:51.311689 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3a7b1e7-98cf-4d6d-9798-2a3b535d9eaa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:05:51.387641 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-b3a7b1e7-98cf-4d6d-9798-2a3b535d9eaa None None] MFA Rules not processed for user `cc99c93da5074cf5a48bc2aa9a92c5e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 22:05:51.388947 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-b3a7b1e7-98cf-4d6d-9798-2a3b535d9eaa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:05:51.434691 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:05:51.493636 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 716/1436] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jul 13 22:05:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 184 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 22:05:51.645408 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba9bfce-3193-47c6-b530-b417a5c8f570 None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:05:51.647728 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-4ba9bfce-3193-47c6-b530-b417a5c8f570 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:05:51.806390 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba9bfce-3193-47c6-b530-b417a5c8f570 None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:05:51.867595 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:05:51.927667 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48c5eb7c-de8c-4132-971e-abbbfac8d786 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84281091687464990ffbca590b9343b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417e3e2f1333481795a089bef2e24bea', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:05:51.928331 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c5eb7c-de8c-4132-971e-abbbfac8d786 service nova] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:05:51.928611 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c5eb7c-de8c-4132-971e-abbbfac8d786 service nova] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:05:51.928880 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48c5eb7c-de8c-4132-971e-abbbfac8d786 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:05:51.930580 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-48c5eb7c-de8c-4132-971e-abbbfac8d786 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:05:52.038025 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c5eb7c-de8c-4132-971e-abbbfac8d786 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:05:52.039749 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-48c5eb7c-de8c-4132-971e-abbbfac8d786 service nova] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:05:52.058082 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 721/1437] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jul 13 22:05:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 414 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 22:05:52.878008 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ddbc4c-8cca-4ceb-a9d5-81739429118e None None] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:05:52.878909 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ddbc4c-8cca-4ceb-a9d5-81739429118e None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:05:52.878909 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-57ddbc4c-8cca-4ceb-a9d5-81739429118e None None] PATH_INFO: `/` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:05:52.881093 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 717/1438] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jul 13 22:05:52 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 22:05:52.886291 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1700953a-d1cb-406b-899a-4f3a6a05259c None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:05:52.886851 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1700953a-d1cb-406b-899a-4f3a6a05259c None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:05:52.886851 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1700953a-d1cb-406b-899a-4f3a6a05259c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:05:52.974970 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-1700953a-d1cb-406b-899a-4f3a6a05259c None None] MFA Rules not processed for user `cc99c93da5074cf5a48bc2aa9a92c5e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 22:05:52.976221 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-1700953a-d1cb-406b-899a-4f3a6a05259c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:05:52.978428 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:05:53.079324 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 722/1439] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jul 13 22:05:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 22:05:53.232766 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ddbc4c-8cca-4ceb-a9d5-81739429118e None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:05:53.235310 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-57ddbc4c-8cca-4ceb-a9d5-81739429118e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:05:53.394725 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-57ddbc4c-8cca-4ceb-a9d5-81739429118e None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:05:53.423212 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:05:53.528234 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-40158904-b807-4e1d-83eb-2e01f6302b56 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84281091687464990ffbca590b9343b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417e3e2f1333481795a089bef2e24bea', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:05:53.528808 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40158904-b807-4e1d-83eb-2e01f6302b56 service nova] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:05:53.529089 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40158904-b807-4e1d-83eb-2e01f6302b56 service nova] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:05:53.529362 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-40158904-b807-4e1d-83eb-2e01f6302b56 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:05:53.531077 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-40158904-b807-4e1d-83eb-2e01f6302b56 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:05:53.628740 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40158904-b807-4e1d-83eb-2e01f6302b56 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:05:53.630362 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-40158904-b807-4e1d-83eb-2e01f6302b56 service nova] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:05:53.651549 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 718/1440] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jul 13 22:05:53 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 420 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 22:05:55.591056 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44ff5dd-b03c-42b6-8653-85f38535f866 None None] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:05:55.591596 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44ff5dd-b03c-42b6-8653-85f38535f866 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:05:55.591596 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b44ff5dd-b03c-42b6-8653-85f38535f866 None None] PATH_INFO: `/` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:05:55.592984 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 723/1441] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 13 22:05:55 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 22:05:55.598479 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99345023-8b21-4c93-ae9c-8f0fca9a9eb3 None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:05:55.598964 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99345023-8b21-4c93-ae9c-8f0fca9a9eb3 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:05:55.599079 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99345023-8b21-4c93-ae9c-8f0fca9a9eb3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:05:55.640123 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:05:55.672852 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-99345023-8b21-4c93-ae9c-8f0fca9a9eb3 None None] MFA Rules not processed for user `cc99c93da5074cf5a48bc2aa9a92c5e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 22:05:55.674089 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-99345023-8b21-4c93-ae9c-8f0fca9a9eb3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:05:55.775452 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 719/1442] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 13 22:05:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 22:05:55.822960 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44ff5dd-b03c-42b6-8653-85f38535f866 None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:05:55.826180 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-b44ff5dd-b03c-42b6-8653-85f38535f866 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:05:55.949298 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b44ff5dd-b03c-42b6-8653-85f38535f866 None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:05:55.982587 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:05:56.024974 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fdec54e-5ec4-41c2-adef-2d7a6e7ba4bb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '847ec9a1bd4d4dec804318a5c845895a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417e3e2f1333481795a089bef2e24bea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:05:56.025651 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdec54e-5ec4-41c2-adef-2d7a6e7ba4bb service neutron] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:05:56.025929 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdec54e-5ec4-41c2-adef-2d7a6e7ba4bb service neutron] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:05:56.026197 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fdec54e-5ec4-41c2-adef-2d7a6e7ba4bb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:05:56.028122 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-4fdec54e-5ec4-41c2-adef-2d7a6e7ba4bb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:05:56.140378 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fdec54e-5ec4-41c2-adef-2d7a6e7ba4bb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:05:56.141757 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fdec54e-5ec4-41c2-adef-2d7a6e7ba4bb service neutron] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:05:56.160872 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 724/1443] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jul 13 22:05:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 339 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 22:05:58.372772 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152b31d5-2760-422b-8c3f-0991068df27e None None] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:05:58.373661 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152b31d5-2760-422b-8c3f-0991068df27e None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:05:58.373661 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-152b31d5-2760-422b-8c3f-0991068df27e None None] PATH_INFO: `/` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:05:58.375874 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 720/1444] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 13 22:05:58 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 22:05:58.381589 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e23cf3-36fb-4721-af32-172f4a162f71 None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:05:58.382021 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e23cf3-36fb-4721-af32-172f4a162f71 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:05:58.382113 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7e23cf3-36fb-4721-af32-172f4a162f71 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:05:58.580114 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-c7e23cf3-36fb-4721-af32-172f4a162f71 None None] MFA Rules not processed for user `cc99c93da5074cf5a48bc2aa9a92c5e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 22:05:58.581338 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-c7e23cf3-36fb-4721-af32-172f4a162f71 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:05:58.596069 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:05:58.688945 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 725/1445] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 13 22:05:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 308 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 22:05:58.728444 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152b31d5-2760-422b-8c3f-0991068df27e None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:05:58.729980 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-152b31d5-2760-422b-8c3f-0991068df27e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:05:58.842071 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-152b31d5-2760-422b-8c3f-0991068df27e None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:05:58.854506 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:05:58.917009 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d7317dd-efef-41e3-a3f2-9fab4749d9e3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '847ec9a1bd4d4dec804318a5c845895a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417e3e2f1333481795a089bef2e24bea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:05:58.917656 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7317dd-efef-41e3-a3f2-9fab4749d9e3 service neutron] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:05:58.917934 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7317dd-efef-41e3-a3f2-9fab4749d9e3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:05:58.918213 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d7317dd-efef-41e3-a3f2-9fab4749d9e3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:05:58.919906 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-5d7317dd-efef-41e3-a3f2-9fab4749d9e3 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:05:59.040845 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7317dd-efef-41e3-a3f2-9fab4749d9e3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:05:59.042195 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d7317dd-efef-41e3-a3f2-9fab4749d9e3 service neutron] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:05:59.061397 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 721/1446] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jul 13 22:05:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 13 22:06:01.562350 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748c6608-e557-4cb2-89ae-51bc58d453e8 None None] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:06:01.563155 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748c6608-e557-4cb2-89ae-51bc58d453e8 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:06:01.563310 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-748c6608-e557-4cb2-89ae-51bc58d453e8 None None] PATH_INFO: `/` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:06:01.565348 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 726/1447] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 13 22:06:01 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 22:06:01.572786 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f38f31c-2d17-4615-9c2c-60ff40bc0ee0 None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:06:01.573566 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f38f31c-2d17-4615-9c2c-60ff40bc0ee0 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:06:01.573692 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f38f31c-2d17-4615-9c2c-60ff40bc0ee0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:06:01.620842 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:06:01.639567 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-2f38f31c-2d17-4615-9c2c-60ff40bc0ee0 None None] MFA Rules not processed for user `cc99c93da5074cf5a48bc2aa9a92c5e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 22:06:01.640793 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-2f38f31c-2d17-4615-9c2c-60ff40bc0ee0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:06:01.748185 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 722/1448] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 13 22:06:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 176 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 22:06:01.794766 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748c6608-e557-4cb2-89ae-51bc58d453e8 None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:06:01.796233 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-748c6608-e557-4cb2-89ae-51bc58d453e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:06:01.908488 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-748c6608-e557-4cb2-89ae-51bc58d453e8 None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:06:01.955321 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:06:01.997809 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ea0078-9c4e-474c-afd5-68ba5c58f4b9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '847ec9a1bd4d4dec804318a5c845895a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417e3e2f1333481795a089bef2e24bea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:06:01.998464 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ea0078-9c4e-474c-afd5-68ba5c58f4b9 service neutron] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:06:01.998773 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ea0078-9c4e-474c-afd5-68ba5c58f4b9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:06:01.999039 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ea0078-9c4e-474c-afd5-68ba5c58f4b9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:06:02.000678 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-d0ea0078-9c4e-474c-afd5-68ba5c58f4b9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:06:02.118978 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ea0078-9c4e-474c-afd5-68ba5c58f4b9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:06:02.120320 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ea0078-9c4e-474c-afd5-68ba5c58f4b9 service neutron] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:06:02.139087 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 727/1449] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jul 13 22:06:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 13 22:06:04.276684 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c4247e-4954-4976-b774-0ce690778379 None None] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:06:04.277307 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c4247e-4954-4976-b774-0ce690778379 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:06:04.277307 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5c4247e-4954-4976-b774-0ce690778379 None None] PATH_INFO: `/` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:06:04.278670 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 723/1450] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 13 22:06:04 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 22:06:04.285049 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c547dcb7-4bca-45cc-a296-d982332b222b None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:06:04.285766 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c547dcb7-4bca-45cc-a296-d982332b222b None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:06:04.285950 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c547dcb7-4bca-45cc-a296-d982332b222b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:06:04.367249 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-c547dcb7-4bca-45cc-a296-d982332b222b None None] MFA Rules not processed for user `cc99c93da5074cf5a48bc2aa9a92c5e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 22:06:04.368553 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-c547dcb7-4bca-45cc-a296-d982332b222b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:06:04.390271 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:06:04.472069 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 728/1451] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 13 22:06:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 22:06:04.513002 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c4247e-4954-4976-b774-0ce690778379 None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:06:04.514518 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-f5c4247e-4954-4976-b774-0ce690778379 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:06:04.626828 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5c4247e-4954-4976-b774-0ce690778379 None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:06:04.654705 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:06:04.715677 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4fca97f0-8d75-4140-ab65-b80f7553ddc2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '847ec9a1bd4d4dec804318a5c845895a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417e3e2f1333481795a089bef2e24bea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:06:04.716327 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fca97f0-8d75-4140-ab65-b80f7553ddc2 service neutron] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:06:04.716598 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fca97f0-8d75-4140-ab65-b80f7553ddc2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:06:04.716863 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4fca97f0-8d75-4140-ab65-b80f7553ddc2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:06:04.718529 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-4fca97f0-8d75-4140-ab65-b80f7553ddc2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:06:04.845505 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fca97f0-8d75-4140-ab65-b80f7553ddc2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:06:04.846986 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4fca97f0-8d75-4140-ab65-b80f7553ddc2 service neutron] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:06:04.865447 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 724/1452] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jul 13 22:06:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 353 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 22:06:07.062515 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b874c2d8-4c1b-493c-aad8-92626b7dab2f None None] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:06:07.063093 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b874c2d8-4c1b-493c-aad8-92626b7dab2f None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:06:07.063093 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b874c2d8-4c1b-493c-aad8-92626b7dab2f None None] PATH_INFO: `/` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:06:07.064440 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 729/1453] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 13 22:06:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 22:06:07.070905 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7dc698a-f92d-459c-8ff0-3fc23c853569 None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:06:07.072075 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7dc698a-f92d-459c-8ff0-3fc23c853569 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:06:07.072075 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7dc698a-f92d-459c-8ff0-3fc23c853569 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:06:07.145479 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:06:07.148511 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-b7dc698a-f92d-459c-8ff0-3fc23c853569 None None] MFA Rules not processed for user `cc99c93da5074cf5a48bc2aa9a92c5e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 22:06:07.150290 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-b7dc698a-f92d-459c-8ff0-3fc23c853569 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:06:07.264821 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 725/1454] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 13 22:06:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 22:06:07.305733 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b874c2d8-4c1b-493c-aad8-92626b7dab2f None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:06:07.307605 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-b874c2d8-4c1b-493c-aad8-92626b7dab2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:06:07.423579 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b874c2d8-4c1b-493c-aad8-92626b7dab2f None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:06:07.480788 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:06:07.507805 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cb17223-ade8-4d64-a7b0-3c19b29c3d40 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '847ec9a1bd4d4dec804318a5c845895a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417e3e2f1333481795a089bef2e24bea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:06:07.508478 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb17223-ade8-4d64-a7b0-3c19b29c3d40 service neutron] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:06:07.508773 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb17223-ade8-4d64-a7b0-3c19b29c3d40 service neutron] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:06:07.509052 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cb17223-ade8-4d64-a7b0-3c19b29c3d40 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:06:07.510727 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-0cb17223-ade8-4d64-a7b0-3c19b29c3d40 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:06:07.627388 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb17223-ade8-4d64-a7b0-3c19b29c3d40 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:06:07.628837 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cb17223-ade8-4d64-a7b0-3c19b29c3d40 service neutron] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:06:07.647704 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 730/1455] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jul 13 22:06:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 22:06:09.804540 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219637d1-9731-43f3-9560-5cc543dcea2b None None] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:06:09.805087 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219637d1-9731-43f3-9560-5cc543dcea2b None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:06:09.805087 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-219637d1-9731-43f3-9560-5cc543dcea2b None None] PATH_INFO: `/` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:06:09.806868 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 726/1456] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 13 22:06:09 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 22:06:09.815270 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df92be6-5f86-4301-930a-5bfe595644d3 None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:06:09.815752 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df92be6-5f86-4301-930a-5bfe595644d3 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:06:09.815855 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9df92be6-5f86-4301-930a-5bfe595644d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:06:09.885139 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-9df92be6-5f86-4301-930a-5bfe595644d3 None None] MFA Rules not processed for user `cc99c93da5074cf5a48bc2aa9a92c5e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 22:06:09.886334 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-9df92be6-5f86-4301-930a-5bfe595644d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:06:09.917396 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:06:09.991636 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 731/1457] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 13 22:06:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 177 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 13 22:06:10.029659 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219637d1-9731-43f3-9560-5cc543dcea2b None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:06:10.031405 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-219637d1-9731-43f3-9560-5cc543dcea2b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:06:10.151686 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-219637d1-9731-43f3-9560-5cc543dcea2b None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:06:10.182175 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:06:10.232221 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df9124e-7e74-418a-9b52-b6a90d400451 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '847ec9a1bd4d4dec804318a5c845895a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417e3e2f1333481795a089bef2e24bea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:06:10.232875 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df9124e-7e74-418a-9b52-b6a90d400451 service neutron] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:06:10.233148 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df9124e-7e74-418a-9b52-b6a90d400451 service neutron] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:06:10.233419 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df9124e-7e74-418a-9b52-b6a90d400451 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:06:10.235107 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-3df9124e-7e74-418a-9b52-b6a90d400451 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:06:10.349074 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df9124e-7e74-418a-9b52-b6a90d400451 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:06:10.350594 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df9124e-7e74-418a-9b52-b6a90d400451 service neutron] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:06:10.370787 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 727/1458] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jul 13 22:06:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 342 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 22:06:12.412195 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bd8761-9f56-44e8-bfb6-9122e68578b8 None None] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:06:12.413027 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bd8761-9f56-44e8-bfb6-9122e68578b8 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:06:12.413027 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00bd8761-9f56-44e8-bfb6-9122e68578b8 None None] PATH_INFO: `/` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:06:12.415373 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 732/1459] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 13 22:06:12 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 22:06:12.421988 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f72c4e-3ecf-4ccd-9f7b-6b31736d9ece None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:06:12.422818 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f72c4e-3ecf-4ccd-9f7b-6b31736d9ece None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:06:12.422987 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41f72c4e-3ecf-4ccd-9f7b-6b31736d9ece None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:06:12.493763 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-41f72c4e-3ecf-4ccd-9f7b-6b31736d9ece None None] MFA Rules not processed for user `cc99c93da5074cf5a48bc2aa9a92c5e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 22:06:12.495088 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-41f72c4e-3ecf-4ccd-9f7b-6b31736d9ece None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:06:12.507251 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:06:12.609578 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 728/1460] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 13 22:06:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 189 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 22:06:12.650818 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00bd8761-9f56-44e8-bfb6-9122e68578b8 None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:06:12.652417 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-00bd8761-9f56-44e8-bfb6-9122e68578b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:06:12.772606 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00bd8761-9f56-44e8-bfb6-9122e68578b8 None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:06:12.830914 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:06:12.851925 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8028c18-3471-43c7-9811-7dbd64c43adf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '847ec9a1bd4d4dec804318a5c845895a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417e3e2f1333481795a089bef2e24bea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:06:12.852652 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8028c18-3471-43c7-9811-7dbd64c43adf service neutron] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:06:12.852931 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8028c18-3471-43c7-9811-7dbd64c43adf service neutron] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:06:12.853209 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8028c18-3471-43c7-9811-7dbd64c43adf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:06:12.854903 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-f8028c18-3471-43c7-9811-7dbd64c43adf service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:06:12.978342 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8028c18-3471-43c7-9811-7dbd64c43adf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:06:12.979825 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8028c18-3471-43c7-9811-7dbd64c43adf service neutron] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:06:12.999113 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 733/1461] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jul 13 22:06:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 349 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 22:06:15.043814 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442ed775-f806-4f3e-838e-c23855119e8f None None] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:06:15.044417 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442ed775-f806-4f3e-838e-c23855119e8f None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:06:15.044417 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-442ed775-f806-4f3e-838e-c23855119e8f None None] PATH_INFO: `/` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:06:15.045798 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 729/1462] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 13 22:06:15 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 22:06:15.052287 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a895ce2-4ead-4173-8c40-3865070bce74 None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:06:15.052693 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a895ce2-4ead-4173-8c40-3865070bce74 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:06:15.052809 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a895ce2-4ead-4173-8c40-3865070bce74 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:06:15.125123 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-0a895ce2-4ead-4173-8c40-3865070bce74 None None] MFA Rules not processed for user `cc99c93da5074cf5a48bc2aa9a92c5e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 22:06:15.126344 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-0a895ce2-4ead-4173-8c40-3865070bce74 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:06:15.166257 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:06:15.228763 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 734/1463] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 13 22:06:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 178 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 22:06:15.266793 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442ed775-f806-4f3e-838e-c23855119e8f None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:06:15.268422 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-442ed775-f806-4f3e-838e-c23855119e8f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:06:15.376105 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-442ed775-f806-4f3e-838e-c23855119e8f None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:06:15.414600 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:06:15.460795 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dd840a2-595e-4cdf-ba39-982c92d6f3bb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '847ec9a1bd4d4dec804318a5c845895a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417e3e2f1333481795a089bef2e24bea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:06:15.461681 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd840a2-595e-4cdf-ba39-982c92d6f3bb service neutron] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:06:15.461962 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd840a2-595e-4cdf-ba39-982c92d6f3bb service neutron] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:06:15.462225 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dd840a2-595e-4cdf-ba39-982c92d6f3bb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:06:15.463907 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-4dd840a2-595e-4cdf-ba39-982c92d6f3bb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:06:15.578936 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd840a2-595e-4cdf-ba39-982c92d6f3bb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:06:15.580397 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dd840a2-595e-4cdf-ba39-982c92d6f3bb service neutron] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:06:15.599272 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 730/1464] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jul 13 22:06:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 333 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 22:06:17.680529 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098b3c98-b564-4d30-a01d-4a9f73ff1ac9 None None] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:06:17.681112 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098b3c98-b564-4d30-a01d-4a9f73ff1ac9 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:06:17.681112 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-098b3c98-b564-4d30-a01d-4a9f73ff1ac9 None None] PATH_INFO: `/` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:06:17.682445 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 735/1465] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 13 22:06:17 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 22:06:17.687940 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5274732e-314e-498d-a1a9-150152735745 None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:06:17.688388 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5274732e-314e-498d-a1a9-150152735745 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:06:17.688479 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5274732e-314e-498d-a1a9-150152735745 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:06:17.757126 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-5274732e-314e-498d-a1a9-150152735745 None None] MFA Rules not processed for user `cc99c93da5074cf5a48bc2aa9a92c5e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 22:06:17.758396 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-5274732e-314e-498d-a1a9-150152735745 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:06:17.778539 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:06:17.860927 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 731/1466] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 13 22:06:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 174 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 22:06:17.899933 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098b3c98-b564-4d30-a01d-4a9f73ff1ac9 None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:06:17.901474 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-098b3c98-b564-4d30-a01d-4a9f73ff1ac9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:06:18.016883 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-098b3c98-b564-4d30-a01d-4a9f73ff1ac9 None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:06:18.093629 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:06:18.103652 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0732b3ee-aa1f-4992-aa7f-14d79b625b7d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '847ec9a1bd4d4dec804318a5c845895a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417e3e2f1333481795a089bef2e24bea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:06:18.104310 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0732b3ee-aa1f-4992-aa7f-14d79b625b7d service neutron] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:06:18.104585 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0732b3ee-aa1f-4992-aa7f-14d79b625b7d service neutron] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:06:18.104851 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0732b3ee-aa1f-4992-aa7f-14d79b625b7d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:06:18.106586 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-0732b3ee-aa1f-4992-aa7f-14d79b625b7d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:06:18.224299 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0732b3ee-aa1f-4992-aa7f-14d79b625b7d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:06:18.225745 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0732b3ee-aa1f-4992-aa7f-14d79b625b7d service neutron] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:06:18.245117 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 736/1467] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jul 13 22:06:17 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 346 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 22:06:20.293495 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603cb087-16ac-4471-9688-e933cc0a923b None None] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:06:20.294446 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603cb087-16ac-4471-9688-e933cc0a923b None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:06:20.294446 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-603cb087-16ac-4471-9688-e933cc0a923b None None] PATH_INFO: `/` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:06:20.296804 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 732/1468] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 13 22:06:20 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 22:06:20.302420 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b322b20-66f6-45ec-8a67-33f9e4968bc3 None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:06:20.302955 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b322b20-66f6-45ec-8a67-33f9e4968bc3 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:06:20.303063 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b322b20-66f6-45ec-8a67-33f9e4968bc3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:06:20.370420 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-7b322b20-66f6-45ec-8a67-33f9e4968bc3 None None] MFA Rules not processed for user `cc99c93da5074cf5a48bc2aa9a92c5e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 22:06:20.371691 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-7b322b20-66f6-45ec-8a67-33f9e4968bc3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:06:20.422400 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:06:20.476557 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 737/1469] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 13 22:06:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 13 22:06:20.513519 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603cb087-16ac-4471-9688-e933cc0a923b None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:06:20.515172 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-603cb087-16ac-4471-9688-e933cc0a923b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:06:20.633088 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-603cb087-16ac-4471-9688-e933cc0a923b None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:06:20.683545 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:06:20.710941 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8f0b61d8-89da-4f26-9655-40260d1df886 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '847ec9a1bd4d4dec804318a5c845895a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417e3e2f1333481795a089bef2e24bea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:06:20.711671 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0b61d8-89da-4f26-9655-40260d1df886 service neutron] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:06:20.712131 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0b61d8-89da-4f26-9655-40260d1df886 service neutron] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:06:20.712412 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f0b61d8-89da-4f26-9655-40260d1df886 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:06:20.714057 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-8f0b61d8-89da-4f26-9655-40260d1df886 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:06:20.825689 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0b61d8-89da-4f26-9655-40260d1df886 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:06:20.827097 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8f0b61d8-89da-4f26-9655-40260d1df886 service neutron] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:06:20.846920 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 733/1470] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jul 13 22:06:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 22:06:30.474337 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=22 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:06:30.844344 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=66 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:06:40.605963 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101c36a2-4b1c-42a4-81d8-66f05a2eac74 None None] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:06:40.606590 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101c36a2-4b1c-42a4-81d8-66f05a2eac74 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:06:40.606590 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-101c36a2-4b1c-42a4-81d8-66f05a2eac74 None None] PATH_INFO: `/` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:06:40.608061 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 738/1471] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jul 13 22:06:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 22:06:40.613523 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc05ae01-712a-4798-8bbf-c6ce97ce7497 None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:06:40.614009 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc05ae01-712a-4798-8bbf-c6ce97ce7497 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:06:40.614110 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc05ae01-712a-4798-8bbf-c6ce97ce7497 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:06:40.681119 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-dc05ae01-712a-4798-8bbf-c6ce97ce7497 None None] MFA Rules not processed for user `cc99c93da5074cf5a48bc2aa9a92c5e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 22:06:40.682327 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-dc05ae01-712a-4798-8bbf-c6ce97ce7497 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:06:40.782442 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 734/1472] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jul 13 22:06:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 170 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 22:06:40.876690 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83165b5c-bfed-4ac4-a8f1-1da2a503d1c1 None None] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:06:40.876976 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83165b5c-bfed-4ac4-a8f1-1da2a503d1c1 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:06:40.877250 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83165b5c-bfed-4ac4-a8f1-1da2a503d1c1 None None] PATH_INFO: `/` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:06:40.878680 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 739/1473] 10.222.0.52 () {54 vars in 962 bytes} [Wed Jul 13 22:06:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 22:06:40.885521 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae650559-460f-4f32-bc37-62c03081fb97 None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:06:40.885809 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae650559-460f-4f32-bc37-62c03081fb97 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:06:40.886086 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae650559-460f-4f32-bc37-62c03081fb97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:06:40.969062 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-ae650559-460f-4f32-bc37-62c03081fb97 None None] MFA Rules not processed for user `e4d35d50e15d43d3afc59206b8165f30`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 22:06:40.970325 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-ae650559-460f-4f32-bc37-62c03081fb97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:06:40.992849 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:06:41.051651 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 735/1474] 10.222.0.52 () {58 vars in 1059 bytes} [Wed Jul 13 22:06:40 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 167 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 22:06:41.059566 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83165b5c-bfed-4ac4-a8f1-1da2a503d1c1 None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:06:41.061084 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-83165b5c-bfed-4ac4-a8f1-1da2a503d1c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:06:41.170850 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83165b5c-bfed-4ac4-a8f1-1da2a503d1c1 None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:06:41.244839 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6bb33d-bb4a-40b3-ab90-7aaf473418e7 service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4d35d50e15d43d3afc59206b8165f30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417e3e2f1333481795a089bef2e24bea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:06:41.245508 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6bb33d-bb4a-40b3-ab90-7aaf473418e7 service glance] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:06:41.245787 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6bb33d-bb4a-40b3-ab90-7aaf473418e7 service glance] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:06:41.246064 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6bb33d-bb4a-40b3-ab90-7aaf473418e7 service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:06:41.247753 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-ff6bb33d-bb4a-40b3-ab90-7aaf473418e7 service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:06:41.305241 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:06:41.374259 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6bb33d-bb4a-40b3-ab90-7aaf473418e7 service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:06:41.375761 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6bb33d-bb4a-40b3-ab90-7aaf473418e7 service glance] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:06:41.395791 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 740/1475] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jul 13 22:06:41 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 337 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 22:06:43.519291 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973e6ee0-7b5a-4658-a86e-3086d5821f5e None None] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:06:43.519907 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973e6ee0-7b5a-4658-a86e-3086d5821f5e None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:06:43.519907 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-973e6ee0-7b5a-4658-a86e-3086d5821f5e None None] PATH_INFO: `/` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:06:43.521248 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 736/1476] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jul 13 22:06:43 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 22:06:43.526790 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879d35cc-3b82-4d74-b65c-05b4c70868ef None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:06:43.527248 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879d35cc-3b82-4d74-b65c-05b4c70868ef None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:06:43.527379 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-879d35cc-3b82-4d74-b65c-05b4c70868ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:06:43.596521 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-879d35cc-3b82-4d74-b65c-05b4c70868ef None None] MFA Rules not processed for user `cc99c93da5074cf5a48bc2aa9a92c5e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 22:06:43.597711 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-879d35cc-3b82-4d74-b65c-05b4c70868ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:06:43.700642 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 741/1477] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jul 13 22:06:43 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 175 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 22:06:43.724823 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973e6ee0-7b5a-4658-a86e-3086d5821f5e None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:06:43.726424 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-973e6ee0-7b5a-4658-a86e-3086d5821f5e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:06:43.868947 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-973e6ee0-7b5a-4658-a86e-3086d5821f5e None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:06:43.921583 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:06:43.969320 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-548af503-e527-4111-a757-01dfc32bd0aa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:06:43.969991 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548af503-e527-4111-a757-01dfc32bd0aa admin admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:06:43.970274 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548af503-e527-4111-a757-01dfc32bd0aa admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:06:43.970588 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-548af503-e527-4111-a757-01dfc32bd0aa admin admin] PATH_INFO: `/v3/projects` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:06:43.971338 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548af503-e527-4111-a757-01dfc32bd0aa admin admin] RBAC: Authorizing `identity:list_projects()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:06:43.973168 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: WARNING py.warnings [None req-548af503-e527-4111-a757-01dfc32bd0aa admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_projects": "(role:reader and system_scope:all) or (role:reader and domain_id:%(target.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 13 22:06:43.973168 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: warnings.warn(msg) Jul 13 22:06:43.973168 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: Jul 13 22:06:43.974327 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-548af503-e527-4111-a757-01dfc32bd0aa admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:06:43.993790 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 737/1478] 10.222.0.52 () {56 vars in 1108 bytes} [Wed Jul 13 22:06:43 2022] GET /identity/v3/projects => generated 2553 bytes in 269 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 22:06:46.044419 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283ccf5a-2187-4f72-9575-8648b9e0d1b1 None None] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:06:46.044986 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283ccf5a-2187-4f72-9575-8648b9e0d1b1 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:06:46.044986 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-283ccf5a-2187-4f72-9575-8648b9e0d1b1 None None] PATH_INFO: `/` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:06:46.046361 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 742/1479] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jul 13 22:06:46 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 22:06:46.052918 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab01fdaa-24a7-4f8b-914d-f5dfc3782728 None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:06:46.053611 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab01fdaa-24a7-4f8b-914d-f5dfc3782728 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:06:46.053866 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab01fdaa-24a7-4f8b-914d-f5dfc3782728 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:06:46.126888 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-ab01fdaa-24a7-4f8b-914d-f5dfc3782728 None None] MFA Rules not processed for user `cc99c93da5074cf5a48bc2aa9a92c5e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 22:06:46.128116 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-ab01fdaa-24a7-4f8b-914d-f5dfc3782728 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:06:46.246869 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 738/1480] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jul 13 22:06:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 22:06:46.299535 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283ccf5a-2187-4f72-9575-8648b9e0d1b1 None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:06:46.301059 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-283ccf5a-2187-4f72-9575-8648b9e0d1b1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:06:46.311281 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:06:46.461936 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-283ccf5a-2187-4f72-9575-8648b9e0d1b1 None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:06:46.554684 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:06:46.593620 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c360820d-a92c-474b-908f-dad5058aa317 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84281091687464990ffbca590b9343b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417e3e2f1333481795a089bef2e24bea', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:06:46.594279 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c360820d-a92c-474b-908f-dad5058aa317 service nova] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:06:46.594586 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c360820d-a92c-474b-908f-dad5058aa317 service nova] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:06:46.594858 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c360820d-a92c-474b-908f-dad5058aa317 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:06:46.596517 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-c360820d-a92c-474b-908f-dad5058aa317 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:06:46.692979 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c360820d-a92c-474b-908f-dad5058aa317 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:06:46.694702 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c360820d-a92c-474b-908f-dad5058aa317 service nova] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:06:46.713263 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 743/1481] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jul 13 22:06:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 414 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 22:06:48.890460 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffc7dd7-a579-4576-a9c4-90e5eee122b5 None None] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:06:48.891030 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffc7dd7-a579-4576-a9c4-90e5eee122b5 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:06:48.891030 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ffc7dd7-a579-4576-a9c4-90e5eee122b5 None None] PATH_INFO: `/` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:06:48.892344 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 739/1482] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jul 13 22:06:48 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 22:06:48.898864 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d15478c-2b5d-4e4b-b763-cbe067c72dcb None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:06:48.898864 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d15478c-2b5d-4e4b-b763-cbe067c72dcb None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:06:48.899615 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d15478c-2b5d-4e4b-b763-cbe067c72dcb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:06:48.973144 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-0d15478c-2b5d-4e4b-b763-cbe067c72dcb None None] MFA Rules not processed for user `cc99c93da5074cf5a48bc2aa9a92c5e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 22:06:48.974371 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-0d15478c-2b5d-4e4b-b763-cbe067c72dcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:06:49.023799 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:06:49.076729 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 744/1483] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jul 13 22:06:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 22:06:49.112861 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffc7dd7-a579-4576-a9c4-90e5eee122b5 None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:06:49.114427 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-7ffc7dd7-a579-4576-a9c4-90e5eee122b5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:06:49.140536 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:06:49.231334 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ffc7dd7-a579-4576-a9c4-90e5eee122b5 None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:06:49.311798 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06ff78cf-36a4-4a20-ab4a-a988471de90e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '847ec9a1bd4d4dec804318a5c845895a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417e3e2f1333481795a089bef2e24bea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:06:49.312467 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ff78cf-36a4-4a20-ab4a-a988471de90e service neutron] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:06:49.312751 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ff78cf-36a4-4a20-ab4a-a988471de90e service neutron] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:06:49.313025 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06ff78cf-36a4-4a20-ab4a-a988471de90e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:06:49.314741 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-06ff78cf-36a4-4a20-ab4a-a988471de90e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:06:49.398382 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:06:49.435949 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ff78cf-36a4-4a20-ab4a-a988471de90e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:06:49.437395 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06ff78cf-36a4-4a20-ab4a-a988471de90e service neutron] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:06:49.457618 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 740/1484] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jul 13 22:06:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 345 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 22:06:51.533085 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4536f4-d210-4385-8390-33c8a9790aec None None] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:06:51.534041 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4536f4-d210-4385-8390-33c8a9790aec None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:06:51.534041 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad4536f4-d210-4385-8390-33c8a9790aec None None] PATH_INFO: `/` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:06:51.536356 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 745/1485] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jul 13 22:06:51 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 22:06:51.544501 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a544b75f-30e5-4c2f-ba84-57974756a4ec None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:06:51.545000 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a544b75f-30e5-4c2f-ba84-57974756a4ec None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:06:51.545150 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a544b75f-30e5-4c2f-ba84-57974756a4ec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:06:51.613409 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-a544b75f-30e5-4c2f-ba84-57974756a4ec None None] MFA Rules not processed for user `cc99c93da5074cf5a48bc2aa9a92c5e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 22:06:51.614659 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-a544b75f-30e5-4c2f-ba84-57974756a4ec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:06:51.712784 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 741/1486] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jul 13 22:06:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 169 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 13 22:06:51.754464 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4536f4-d210-4385-8390-33c8a9790aec None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:06:51.756033 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-ad4536f4-d210-4385-8390-33c8a9790aec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:06:51.874634 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad4536f4-d210-4385-8390-33c8a9790aec None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:06:51.950511 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:06:51.953001 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0cbd9b79-cc53-4ef1-b3d4-83543303776d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '847ec9a1bd4d4dec804318a5c845895a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417e3e2f1333481795a089bef2e24bea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:06:51.953642 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbd9b79-cc53-4ef1-b3d4-83543303776d service neutron] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:06:51.953922 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbd9b79-cc53-4ef1-b3d4-83543303776d service neutron] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:06:51.954200 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0cbd9b79-cc53-4ef1-b3d4-83543303776d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:06:51.956658 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-0cbd9b79-cc53-4ef1-b3d4-83543303776d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:06:52.071950 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cbd9b79-cc53-4ef1-b3d4-83543303776d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:06:52.073674 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0cbd9b79-cc53-4ef1-b3d4-83543303776d service neutron] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:06:52.097020 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 746/1487] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jul 13 22:06:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 343 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 22:06:54.201203 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ad4ce2-5549-4560-837c-dc55f2d0f112 None None] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:06:54.201849 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ad4ce2-5549-4560-837c-dc55f2d0f112 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:06:54.201849 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ad4ce2-5549-4560-837c-dc55f2d0f112 None None] PATH_INFO: `/` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:06:54.203192 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 742/1488] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jul 13 22:06:54 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 13 22:06:54.209644 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5244e26b-6956-4668-bc2f-0fb16a365d0c None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:06:54.210098 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5244e26b-6956-4668-bc2f-0fb16a365d0c None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:06:54.210208 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5244e26b-6956-4668-bc2f-0fb16a365d0c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:06:54.293357 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-5244e26b-6956-4668-bc2f-0fb16a365d0c None None] MFA Rules not processed for user `cc99c93da5074cf5a48bc2aa9a92c5e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 22:06:54.294603 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-5244e26b-6956-4668-bc2f-0fb16a365d0c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:06:54.350189 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:06:54.403220 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 747/1489] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jul 13 22:06:54 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 196 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 22:06:54.444051 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ad4ce2-5549-4560-837c-dc55f2d0f112 None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:06:54.446995 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-65ad4ce2-5549-4560-837c-dc55f2d0f112 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:06:54.490188 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:06:54.569471 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ad4ce2-5549-4560-837c-dc55f2d0f112 None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:06:54.657485 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a9c1fb9-ae08-45a8-9bc4-3d47dc68079f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '847ec9a1bd4d4dec804318a5c845895a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417e3e2f1333481795a089bef2e24bea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:06:54.658528 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9c1fb9-ae08-45a8-9bc4-3d47dc68079f service neutron] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:06:54.658992 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9c1fb9-ae08-45a8-9bc4-3d47dc68079f service neutron] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:06:54.659431 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9c1fb9-ae08-45a8-9bc4-3d47dc68079f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:06:54.662049 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-6a9c1fb9-ae08-45a8-9bc4-3d47dc68079f service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:06:54.771559 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:06:54.801452 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9c1fb9-ae08-45a8-9bc4-3d47dc68079f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:06:54.802954 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a9c1fb9-ae08-45a8-9bc4-3d47dc68079f service neutron] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:06:54.822243 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 743/1490] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jul 13 22:06:54 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 379 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 22:06:57.038970 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a3d1ad-97c8-489c-afc4-836a63ae9e56 None None] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:06:57.039890 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a3d1ad-97c8-489c-afc4-836a63ae9e56 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:06:57.039890 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a3d1ad-97c8-489c-afc4-836a63ae9e56 None None] PATH_INFO: `/` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:06:57.042125 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 748/1491] 10.222.0.52 () {54 vars in 926 bytes} [Wed Jul 13 22:06:57 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 22:06:57.051077 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0846a4c-3e84-4436-b833-7abe1b9c562a None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:06:57.051809 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0846a4c-3e84-4436-b833-7abe1b9c562a None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:06:57.051973 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0846a4c-3e84-4436-b833-7abe1b9c562a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:06:57.126423 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-b0846a4c-3e84-4436-b833-7abe1b9c562a None None] MFA Rules not processed for user `cc99c93da5074cf5a48bc2aa9a92c5e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 22:06:57.127660 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-b0846a4c-3e84-4436-b833-7abe1b9c562a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:06:57.227705 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 744/1492] 10.222.0.52 () {58 vars in 1023 bytes} [Wed Jul 13 22:06:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 179 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 22:06:57.271788 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a3d1ad-97c8-489c-afc4-836a63ae9e56 None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:06:57.274224 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-91a3d1ad-97c8-489c-afc4-836a63ae9e56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:06:57.396208 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a3d1ad-97c8-489c-afc4-836a63ae9e56 None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:06:57.475317 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-262ae123-3ad0-4428-9c34-b204d3cadfe4 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '847ec9a1bd4d4dec804318a5c845895a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417e3e2f1333481795a089bef2e24bea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:06:57.475975 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262ae123-3ad0-4428-9c34-b204d3cadfe4 service neutron] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:06:57.476266 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262ae123-3ad0-4428-9c34-b204d3cadfe4 service neutron] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:06:57.476559 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-262ae123-3ad0-4428-9c34-b204d3cadfe4 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:06:57.478241 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-262ae123-3ad0-4428-9c34-b204d3cadfe4 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:06:57.488141 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:06:57.597207 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-262ae123-3ad0-4428-9c34-b204d3cadfe4 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:06:57.598688 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-262ae123-3ad0-4428-9c34-b204d3cadfe4 service neutron] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:06:57.617577 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 749/1493] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jul 13 22:06:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 22:07:07.225483 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=82 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:07:07.615085 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=50 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:08:23.994185 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b65acec-2dc1-46a0-b940-94d118998091 None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:08:23.994185 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b65acec-2dc1-46a0-b940-94d118998091 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:08:23.994834 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b65acec-2dc1-46a0-b940-94d118998091 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:08:24.070404 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-2b65acec-2dc1-46a0-b940-94d118998091 None None] MFA Rules not processed for user `cc99c93da5074cf5a48bc2aa9a92c5e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 22:08:24.071637 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-2b65acec-2dc1-46a0-b940-94d118998091 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:08:24.183040 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 745/1494] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jul 13 22:08:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 190 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 22:08:24.191590 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91523f43-4e0a-4ba0-9709-c863655c1dcb None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:08:24.192136 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91523f43-4e0a-4ba0-9709-c863655c1dcb None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:08:24.192136 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91523f43-4e0a-4ba0-9709-c863655c1dcb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:08:24.268448 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-91523f43-4e0a-4ba0-9709-c863655c1dcb None None] MFA Rules not processed for user `cc99c93da5074cf5a48bc2aa9a92c5e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 22:08:24.269606 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-91523f43-4e0a-4ba0-9709-c863655c1dcb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:08:24.371999 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 750/1495] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jul 13 22:08:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 13 22:08:24.376895 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b65acec-2dc1-46a0-b940-94d118998091 None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:08:24.378109 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-2b65acec-2dc1-46a0-b940-94d118998091 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:08:24.390068 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b65acec-2dc1-46a0-b940-94d118998091 None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:08:24.452465 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:08:24.485935 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35922d18-978a-4918-8e6c-e8349ff02909 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:08:24.486655 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35922d18-978a-4918-8e6c-e8349ff02909 admin admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:08:24.486917 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35922d18-978a-4918-8e6c-e8349ff02909 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:08:24.487200 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35922d18-978a-4918-8e6c-e8349ff02909 admin admin] PATH_INFO: `/v3/domains` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:08:24.487930 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35922d18-978a-4918-8e6c-e8349ff02909 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 22:08:24.488229 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35922d18-978a-4918-8e6c-e8349ff02909 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:08:24.489245 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: WARNING py.warnings [None req-35922d18-978a-4918-8e6c-e8349ff02909 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 13 22:08:24.489245 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: warnings.warn(msg) Jul 13 22:08:24.489245 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: Jul 13 22:08:24.490139 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35922d18-978a-4918-8e6c-e8349ff02909 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:08:24.501176 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 746/1496] 10.222.0.52 () {58 vars in 1152 bytes} [Wed Jul 13 22:08:24 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 125 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 22:08:24.510153 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91523f43-4e0a-4ba0-9709-c863655c1dcb None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:08:24.524496 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91523f43-4e0a-4ba0-9709-c863655c1dcb None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:08:24.590531 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:08:24.629298 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b100e5-cb50-4674-b424-95574e57b7a5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:08:24.629966 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b100e5-cb50-4674-b424-95574e57b7a5 admin admin] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:08:24.630241 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b100e5-cb50-4674-b424-95574e57b7a5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:08:24.630532 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81b100e5-cb50-4674-b424-95574e57b7a5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:08:24.631252 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b100e5-cb50-4674-b424-95574e57b7a5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:08:24.632999 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: WARNING py.warnings [None req-81b100e5-cb50-4674-b424-95574e57b7a5 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 13 22:08:24.632999 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: warnings.warn(msg) Jul 13 22:08:24.632999 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: Jul 13 22:08:24.634183 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-81b100e5-cb50-4674-b424-95574e57b7a5 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:08:24.696704 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 751/1497] 10.222.0.52 () {60 vars in 1152 bytes} [Wed Jul 13 22:08:24 2022] POST /identity/v3/projects => generated 379 bytes in 188 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 22:08:24.701394 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35922d18-978a-4918-8e6c-e8349ff02909 admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:08:24.703020 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-35922d18-978a-4918-8e6c-e8349ff02909 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:08:24.743736 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35922d18-978a-4918-8e6c-e8349ff02909 admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:08:24.852239 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff830fe-6dc5-418b-8af3-e2b13b711559 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:08:24.852818 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff830fe-6dc5-418b-8af3-e2b13b711559 admin admin] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:08:24.853113 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff830fe-6dc5-418b-8af3-e2b13b711559 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:08:24.853388 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ff830fe-6dc5-418b-8af3-e2b13b711559 admin admin] PATH_INFO: `/v3/users` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:08:24.854107 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff830fe-6dc5-418b-8af3-e2b13b711559 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:08:24.855144 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: WARNING py.warnings [None req-2ff830fe-6dc5-418b-8af3-e2b13b711559 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 13 22:08:24.855144 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: warnings.warn(msg) Jul 13 22:08:24.855144 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: Jul 13 22:08:24.856338 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ff830fe-6dc5-418b-8af3-e2b13b711559 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:08:24.902576 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 747/1498] 10.222.0.52 () {60 vars in 1143 bytes} [Wed Jul 13 22:08:24 2022] POST /identity/v3/users => generated 290 bytes in 202 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 22:08:24.907871 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b100e5-cb50-4674-b424-95574e57b7a5 admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:08:24.909378 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-81b100e5-cb50-4674-b424-95574e57b7a5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:08:24.975455 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81b100e5-cb50-4674-b424-95574e57b7a5 admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:08:25.063764 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:08:25.075711 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5581c4bc-768b-40c0-bb1f-55be623bbb4d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:08:25.076417 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5581c4bc-768b-40c0-bb1f-55be623bbb4d admin admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:08:25.076691 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5581c4bc-768b-40c0-bb1f-55be623bbb4d admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:08:25.076966 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5581c4bc-768b-40c0-bb1f-55be623bbb4d admin admin] PATH_INFO: `/v3/roles` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:08:25.077696 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5581c4bc-768b-40c0-bb1f-55be623bbb4d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:08:25.078786 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: WARNING py.warnings [None req-5581c4bc-768b-40c0-bb1f-55be623bbb4d admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 13 22:08:25.078786 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: warnings.warn(msg) Jul 13 22:08:25.078786 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: Jul 13 22:08:25.079661 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5581c4bc-768b-40c0-bb1f-55be623bbb4d admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:08:25.092306 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 752/1499] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jul 13 22:08:24 2022] GET /identity/v3/roles => generated 1420 bytes in 185 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 22:08:25.096681 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff830fe-6dc5-418b-8af3-e2b13b711559 admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:08:25.098246 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-2ff830fe-6dc5-418b-8af3-e2b13b711559 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:08:25.116777 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ff830fe-6dc5-418b-8af3-e2b13b711559 admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:08:25.145519 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:08:25.217011 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5065bdf-6ca0-46f1-86d3-71103086e17f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:08:25.217683 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5065bdf-6ca0-46f1-86d3-71103086e17f admin admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:08:25.217959 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5065bdf-6ca0-46f1-86d3-71103086e17f admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:08:25.218230 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5065bdf-6ca0-46f1-86d3-71103086e17f admin admin] PATH_INFO: `/v3/roles` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:08:25.219003 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5065bdf-6ca0-46f1-86d3-71103086e17f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:08:25.220029 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: WARNING py.warnings [None req-f5065bdf-6ca0-46f1-86d3-71103086e17f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_roles": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 13 22:08:25.220029 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: warnings.warn(msg) Jul 13 22:08:25.220029 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: Jul 13 22:08:25.220914 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5065bdf-6ca0-46f1-86d3-71103086e17f admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:08:25.233708 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 748/1500] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jul 13 22:08:25 2022] GET /identity/v3/roles => generated 1420 bytes in 138 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 22:08:25.238637 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5581c4bc-768b-40c0-bb1f-55be623bbb4d admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:08:25.240753 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-5581c4bc-768b-40c0-bb1f-55be623bbb4d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:08:25.276340 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5581c4bc-768b-40c0-bb1f-55be623bbb4d admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:08:25.379330 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4897c19-b695-486c-b9ca-f7d49bdab73f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:08:25.380483 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4897c19-b695-486c-b9ca-f7d49bdab73f admin admin] REQUEST_METHOD: `PUT` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:08:25.380824 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4897c19-b695-486c-b9ca-f7d49bdab73f admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:08:25.381178 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4897c19-b695-486c-b9ca-f7d49bdab73f admin admin] PATH_INFO: `/v3/projects/fe6d29ad21f14d9f8aa7d004d4f85bd3/users/0776808707514651bb7bcc1b0797d7a0/roles/2a5e86976394498abb6310697cdb3f8b` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:08:25.408389 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4897c19-b695-486c-b9ca-f7d49bdab73f admin admin] RBAC: Authorizing `identity:create_grant(project_id=fe6d29ad21f14d9f8aa7d004d4f85bd3, user_id=0776808707514651bb7bcc1b0797d7a0, role_id=2a5e86976394498abb6310697cdb3f8b)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:08:25.409452 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: WARNING py.warnings [None req-e4897c19-b695-486c-b9ca-f7d49bdab73f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 13 22:08:25.409452 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: warnings.warn(msg) Jul 13 22:08:25.409452 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: Jul 13 22:08:25.411436 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4897c19-b695-486c-b9ca-f7d49bdab73f admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:08:25.439647 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 753/1501] 10.222.0.52 () {60 vars in 1482 bytes} [Wed Jul 13 22:08:25 2022] PUT /identity/v3/projects/fe6d29ad21f14d9f8aa7d004d4f85bd3/users/0776808707514651bb7bcc1b0797d7a0/roles/2a5e86976394498abb6310697cdb3f8b => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 22:08:25.452624 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b439005-e2c5-45c9-aa9f-608bcb3b6bf6 None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:08:25.452926 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b439005-e2c5-45c9-aa9f-608bcb3b6bf6 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:08:25.453163 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b439005-e2c5-45c9-aa9f-608bcb3b6bf6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:08:25.512767 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-2b439005-e2c5-45c9-aa9f-608bcb3b6bf6 None None] MFA Rules not processed for user `0776808707514651bb7bcc1b0797d7a0`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 22:08:25.513979 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-2b439005-e2c5-45c9-aa9f-608bcb3b6bf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:08:25.594369 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 749/1502] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jul 13 22:08:25 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 22:08:25.608094 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653d518e-2986-4a6f-92a8-1f42b115aa17 None None] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:08:25.608454 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653d518e-2986-4a6f-92a8-1f42b115aa17 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:08:25.608737 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653d518e-2986-4a6f-92a8-1f42b115aa17 None None] PATH_INFO: `/` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:08:25.610117 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 754/1503] 10.222.0.52 () {54 vars in 963 bytes} [Wed Jul 13 22:08:25 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 22:08:25.615157 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f0d33a-9224-4a32-a9ee-0d1202a96a9d None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:08:25.615445 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f0d33a-9224-4a32-a9ee-0d1202a96a9d None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:08:25.615720 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2f0d33a-9224-4a32-a9ee-0d1202a96a9d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:08:25.647763 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:08:25.820828 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-d2f0d33a-9224-4a32-a9ee-0d1202a96a9d None None] MFA Rules not processed for user `bc77e1ea1ada4700b809beb1c980c5e1`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 22:08:25.822022 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-d2f0d33a-9224-4a32-a9ee-0d1202a96a9d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:08:25.910739 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 750/1504] 10.222.0.52 () {58 vars in 1060 bytes} [Wed Jul 13 22:08:25 2022] POST /identity/v3/auth/tokens => generated 3110 bytes in 297 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 13 22:08:25.919203 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653d518e-2986-4a6f-92a8-1f42b115aa17 None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:08:25.920731 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-653d518e-2986-4a6f-92a8-1f42b115aa17 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:08:25.988782 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:08:26.043727 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653d518e-2986-4a6f-92a8-1f42b115aa17 None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:08:26.140227 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0736d9-f923-4643-bade-34900b82a466 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc77e1ea1ada4700b809beb1c980c5e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417e3e2f1333481795a089bef2e24bea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:08:26.140903 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0736d9-f923-4643-bade-34900b82a466 service cinder] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:08:26.141180 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0736d9-f923-4643-bade-34900b82a466 service cinder] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:08:26.141457 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d0736d9-f923-4643-bade-34900b82a466 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:08:26.143157 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-3d0736d9-f923-4643-bade-34900b82a466 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:08:26.244658 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0736d9-f923-4643-bade-34900b82a466 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:08:26.246116 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d0736d9-f923-4643-bade-34900b82a466 service cinder] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:08:26.249578 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:08:26.266001 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 755/1505] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jul 13 22:08:25 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 347 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 22:08:26.313792 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f34be56-19dc-4f71-9aa8-a080c574c20b None None] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:08:26.314576 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f34be56-19dc-4f71-9aa8-a080c574c20b None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:08:26.314715 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f34be56-19dc-4f71-9aa8-a080c574c20b None None] PATH_INFO: `/` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:08:26.316823 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 751/1506] 10.222.0.52 () {52 vars in 829 bytes} [Wed Jul 13 22:08:26 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 22:08:26.320675 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0736d9-f923-4643-bade-34900b82a466 service cinder] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:08:26.321898 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-3d0736d9-f923-4643-bade-34900b82a466 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:08:26.430231 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d0736d9-f923-4643-bade-34900b82a466 service cinder] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:08:26.543160 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453eb4e4-acb8-4c3a-b8b8-1c8d4a949a78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:08:26.543956 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453eb4e4-acb8-4c3a-b8b8-1c8d4a949a78 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:08:26.544233 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453eb4e4-acb8-4c3a-b8b8-1c8d4a949a78 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:08:26.544506 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-453eb4e4-acb8-4c3a-b8b8-1c8d4a949a78 admin admin] PATH_INFO: `/v3/users/0776808707514651bb7bcc1b0797d7a0` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:08:26.560810 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453eb4e4-acb8-4c3a-b8b8-1c8d4a949a78 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0776808707514651bb7bcc1b0797d7a0)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:08:26.561866 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: WARNING py.warnings [None req-453eb4e4-acb8-4c3a-b8b8-1c8d4a949a78 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 13 22:08:26.561866 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: warnings.warn(msg) Jul 13 22:08:26.561866 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: Jul 13 22:08:26.563144 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-453eb4e4-acb8-4c3a-b8b8-1c8d4a949a78 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:08:26.574382 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:08:26.636990 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-453eb4e4-acb8-4c3a-b8b8-1c8d4a949a78 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0776808707514651bb7bcc1b0797d7a0'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:08:26.650638 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-453eb4e4-acb8-4c3a-b8b8-1c8d4a949a78 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0776808707514651bb7bcc1b0797d7a0'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:08:26.669508 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-453eb4e4-acb8-4c3a-b8b8-1c8d4a949a78 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0776808707514651bb7bcc1b0797d7a0'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:08:26.669967 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-453eb4e4-acb8-4c3a-b8b8-1c8d4a949a78 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0776808707514651bb7bcc1b0797d7a0'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:08:26.670459 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-453eb4e4-acb8-4c3a-b8b8-1c8d4a949a78 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0776808707514651bb7bcc1b0797d7a0'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:08:26.687976 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 756/1507] 10.222.0.52 () {58 vars in 1223 bytes} [Wed Jul 13 22:08:26 2022] DELETE /identity/v3/users/0776808707514651bb7bcc1b0797d7a0 => generated 0 bytes in 368 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 22:08:26.692627 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59682616-bde6-4398-8c83-5a3be8b2d6b7 None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:08:26.692920 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59682616-bde6-4398-8c83-5a3be8b2d6b7 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:08:26.693173 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59682616-bde6-4398-8c83-5a3be8b2d6b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:08:26.758414 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-59682616-bde6-4398-8c83-5a3be8b2d6b7 None None] MFA Rules not processed for user `cc99c93da5074cf5a48bc2aa9a92c5e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 22:08:26.759669 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-59682616-bde6-4398-8c83-5a3be8b2d6b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:08:26.845352 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:08:26.858077 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 752/1508] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jul 13 22:08:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 22:08:26.867921 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453eb4e4-acb8-4c3a-b8b8-1c8d4a949a78 admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:08:26.869191 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-453eb4e4-acb8-4c3a-b8b8-1c8d4a949a78 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:08:26.975617 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-453eb4e4-acb8-4c3a-b8b8-1c8d4a949a78 admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:08:27.034267 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:DELETE=10,keystone:SELECT=89,keystone:INSERT=1 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:08:27.057685 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d0b3892-c031-4eae-bad4-f62f28c9fabb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '847ec9a1bd4d4dec804318a5c845895a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417e3e2f1333481795a089bef2e24bea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:08:27.058346 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0b3892-c031-4eae-bad4-f62f28c9fabb service neutron] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:08:27.058649 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0b3892-c031-4eae-bad4-f62f28c9fabb service neutron] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:08:27.058922 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d0b3892-c031-4eae-bad4-f62f28c9fabb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:08:27.060566 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-5d0b3892-c031-4eae-bad4-f62f28c9fabb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:08:27.186531 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0b3892-c031-4eae-bad4-f62f28c9fabb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:08:27.187983 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d0b3892-c031-4eae-bad4-f62f28c9fabb service neutron] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:08:27.207921 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 757/1509] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jul 13 22:08:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 341 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 22:08:27.557583 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59682616-bde6-4398-8c83-5a3be8b2d6b7 None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:08:27.558866 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-59682616-bde6-4398-8c83-5a3be8b2d6b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:08:27.571428 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59682616-bde6-4398-8c83-5a3be8b2d6b7 None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:08:27.674165 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fee0286d-6bc7-4449-b4fe-928b865e9b17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:08:27.675082 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee0286d-6bc7-4449-b4fe-928b865e9b17 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:08:27.675359 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee0286d-6bc7-4449-b4fe-928b865e9b17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:08:27.675626 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fee0286d-6bc7-4449-b4fe-928b865e9b17 admin admin] PATH_INFO: `/v3/projects/fe6d29ad21f14d9f8aa7d004d4f85bd3` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:08:27.686172 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee0286d-6bc7-4449-b4fe-928b865e9b17 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fe6d29ad21f14d9f8aa7d004d4f85bd3)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:08:27.687281 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: WARNING py.warnings [None req-fee0286d-6bc7-4449-b4fe-928b865e9b17 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 13 22:08:27.687281 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: warnings.warn(msg) Jul 13 22:08:27.687281 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: Jul 13 22:08:27.688467 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fee0286d-6bc7-4449-b4fe-928b865e9b17 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:08:27.751797 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-fee0286d-6bc7-4449-b4fe-928b865e9b17 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fe6d29ad21f14d9f8aa7d004d4f85bd3', 'request_id': 'req-fee0286d-6bc7-4449-b4fe-928b865e9b17'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:08:27.765427 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-fee0286d-6bc7-4449-b4fe-928b865e9b17 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fe6d29ad21f14d9f8aa7d004d4f85bd3', 'request_id': 'req-fee0286d-6bc7-4449-b4fe-928b865e9b17'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:08:27.771748 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-fee0286d-6bc7-4449-b4fe-928b865e9b17 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fe6d29ad21f14d9f8aa7d004d4f85bd3', 'request_id': 'req-fee0286d-6bc7-4449-b4fe-928b865e9b17'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:08:27.798863 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-fee0286d-6bc7-4449-b4fe-928b865e9b17 admin admin] The token cache is being invalidate because project fe6d29ad21f14d9f8aa7d004d4f85bd3 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80382) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 22:08:27.799131 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-fee0286d-6bc7-4449-b4fe-928b865e9b17 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:08:27.799684 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-fee0286d-6bc7-4449-b4fe-928b865e9b17 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:08:27.801065 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 753/1510] 10.222.0.52 () {58 vars in 1232 bytes} [Wed Jul 13 22:08:27 2022] DELETE /identity/v3/projects/fe6d29ad21f14d9f8aa7d004d4f85bd3 => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 22:08:37.205472 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=62 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:08:37.796665 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=84,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:10:06.037378 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95da3ddd-e5c1-498d-9adc-5ac55dabf288 None None] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:10:06.037984 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95da3ddd-e5c1-498d-9adc-5ac55dabf288 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:10:06.037984 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95da3ddd-e5c1-498d-9adc-5ac55dabf288 None None] PATH_INFO: `/` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:10:06.039337 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 758/1511] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jul 13 22:10:06 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 22:10:06.044418 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e8782e-20b9-40d8-b967-a2276674dd00 None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:10:06.044910 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e8782e-20b9-40d8-b967-a2276674dd00 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:10:06.045015 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a3e8782e-20b9-40d8-b967-a2276674dd00 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:10:06.143717 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-a3e8782e-20b9-40d8-b967-a2276674dd00 None None] MFA Rules not processed for user `cc99c93da5074cf5a48bc2aa9a92c5e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 22:10:06.144927 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-a3e8782e-20b9-40d8-b967-a2276674dd00 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:10:06.250881 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 754/1512] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jul 13 22:10:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 207 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 22:10:06.405880 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95da3ddd-e5c1-498d-9adc-5ac55dabf288 None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:10:06.408916 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-95da3ddd-e5c1-498d-9adc-5ac55dabf288 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:10:06.549689 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95da3ddd-e5c1-498d-9adc-5ac55dabf288 None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:10:06.558877 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28308f71-8d18-4cd6-b2ad-8074beeffece service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84281091687464990ffbca590b9343b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417e3e2f1333481795a089bef2e24bea', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:10:06.559907 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28308f71-8d18-4cd6-b2ad-8074beeffece service nova] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:10:06.560372 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28308f71-8d18-4cd6-b2ad-8074beeffece service nova] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:10:06.560827 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28308f71-8d18-4cd6-b2ad-8074beeffece service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:10:06.570774 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28308f71-8d18-4cd6-b2ad-8074beeffece service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:10:06.572371 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28308f71-8d18-4cd6-b2ad-8074beeffece service nova] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:10:06.582094 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 759/1513] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jul 13 22:10:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 177 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 22:10:07.404330 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1cdc24-07f4-4800-9c8a-d930e2c0cd47 None None] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:10:07.404948 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1cdc24-07f4-4800-9c8a-d930e2c0cd47 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:10:07.404948 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c1cdc24-07f4-4800-9c8a-d930e2c0cd47 None None] PATH_INFO: `/` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:10:07.406349 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 755/1514] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jul 13 22:10:07 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 22:10:07.412407 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae755958-cad0-4c10-8361-e53d115e7a0a None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:10:07.413140 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae755958-cad0-4c10-8361-e53d115e7a0a None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:10:07.413331 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae755958-cad0-4c10-8361-e53d115e7a0a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:10:07.442146 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-ae755958-cad0-4c10-8361-e53d115e7a0a None None] MFA Rules not processed for user `cc99c93da5074cf5a48bc2aa9a92c5e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 22:10:07.443386 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-ae755958-cad0-4c10-8361-e53d115e7a0a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:10:07.454250 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 760/1515] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jul 13 22:10:07 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 22:10:07.611496 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1cdc24-07f4-4800-9c8a-d930e2c0cd47 None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:10:07.628406 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c1cdc24-07f4-4800-9c8a-d930e2c0cd47 None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:10:07.634736 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-977c4c5a-556b-4f27-8216-31646e4e2927 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84281091687464990ffbca590b9343b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417e3e2f1333481795a089bef2e24bea', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:10:07.635378 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977c4c5a-556b-4f27-8216-31646e4e2927 service nova] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:10:07.635651 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977c4c5a-556b-4f27-8216-31646e4e2927 service nova] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:10:07.635925 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-977c4c5a-556b-4f27-8216-31646e4e2927 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:10:07.645298 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977c4c5a-556b-4f27-8216-31646e4e2927 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:10:07.646927 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-977c4c5a-556b-4f27-8216-31646e4e2927 service nova] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:10:07.657501 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 756/1516] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jul 13 22:10:07 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 22:10:08.284603 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7007564f-e1a1-4ee5-b19c-b234bf632638 None None] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:10:08.284855 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7007564f-e1a1-4ee5-b19c-b234bf632638 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:10:08.285064 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7007564f-e1a1-4ee5-b19c-b234bf632638 None None] PATH_INFO: `/v3` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:10:08.286382 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 761/1517] 10.222.0.49 () {54 vars in 925 bytes} [Wed Jul 13 22:10:08 2022] GET /identity/v3 => generated 255 bytes in 3 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 22:10:08.293500 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537757b0-d4c9-4a32-86ad-f5ebe4775bf6 None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:10:08.293791 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537757b0-d4c9-4a32-86ad-f5ebe4775bf6 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:10:08.294080 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537757b0-d4c9-4a32-86ad-f5ebe4775bf6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:10:08.356521 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-537757b0-d4c9-4a32-86ad-f5ebe4775bf6 None None] MFA Rules not processed for user `ac673d641d8849c79bf4cf71c9805587`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 22:10:08.357732 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-537757b0-d4c9-4a32-86ad-f5ebe4775bf6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:10:08.375043 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:10:08.434663 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 757/1518] 10.222.0.49 () {58 vars in 1015 bytes} [Wed Jul 13 22:10:08 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 142 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 22:10:17.438340 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=56 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:10:17.475667 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e14a7c-bc8a-49dc-97ca-702d0d797af5 None None] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:10:17.475850 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e14a7c-bc8a-49dc-97ca-702d0d797af5 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:10:17.476231 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2e14a7c-bc8a-49dc-97ca-702d0d797af5 None None] PATH_INFO: `/` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:10:17.478357 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 762/1519] 10.222.0.52 () {54 vars in 960 bytes} [Wed Jul 13 22:10:17 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 22:10:17.483480 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf70227-279e-4c2f-b5b8-f63b2e4e49a9 None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:10:17.483988 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf70227-279e-4c2f-b5b8-f63b2e4e49a9 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:10:17.483988 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bf70227-279e-4c2f-b5b8-f63b2e4e49a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:10:17.503307 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-3bf70227-279e-4c2f-b5b8-f63b2e4e49a9 None None] MFA Rules not processed for user `ac673d641d8849c79bf4cf71c9805587`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 22:10:17.504555 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-3bf70227-279e-4c2f-b5b8-f63b2e4e49a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:10:17.513669 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 758/1520] 10.222.0.52 () {58 vars in 1057 bytes} [Wed Jul 13 22:10:17 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 22:10:17.521407 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e14a7c-bc8a-49dc-97ca-702d0d797af5 None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:10:17.539504 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2e14a7c-bc8a-49dc-97ca-702d0d797af5 None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:10:17.546983 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c73d1ea-c57e-4dc3-be57-057c0ad9998d service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac673d641d8849c79bf4cf71c9805587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417e3e2f1333481795a089bef2e24bea', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:10:17.548386 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c73d1ea-c57e-4dc3-be57-057c0ad9998d service placement] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:10:17.548666 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c73d1ea-c57e-4dc3-be57-057c0ad9998d service placement] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:10:17.548942 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c73d1ea-c57e-4dc3-be57-057c0ad9998d service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:10:17.557951 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c73d1ea-c57e-4dc3-be57-057c0ad9998d service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:10:17.559596 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c73d1ea-c57e-4dc3-be57-057c0ad9998d service placement] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:10:17.572573 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 763/1521] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jul 13 22:10:17 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 22:10:18.916691 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbeab86-63c0-4197-ade1-b3342c62f2ba None None] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:10:18.917576 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbeab86-63c0-4197-ade1-b3342c62f2ba None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:10:18.917576 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5fbeab86-63c0-4197-ade1-b3342c62f2ba None None] PATH_INFO: `/` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:10:18.919857 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 759/1522] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jul 13 22:10:18 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 22:10:18.925094 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fa346b-1ba0-420e-ab51-802bc9832894 None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:10:18.925542 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fa346b-1ba0-420e-ab51-802bc9832894 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:10:18.925637 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5fa346b-1ba0-420e-ab51-802bc9832894 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:10:18.944666 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-c5fa346b-1ba0-420e-ab51-802bc9832894 None None] MFA Rules not processed for user `cc99c93da5074cf5a48bc2aa9a92c5e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 22:10:18.945875 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-c5fa346b-1ba0-420e-ab51-802bc9832894 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:10:18.955095 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 764/1523] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jul 13 22:10:18 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 22:10:19.110339 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbeab86-63c0-4197-ade1-b3342c62f2ba None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:10:19.131285 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5fbeab86-63c0-4197-ade1-b3342c62f2ba None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:10:19.137354 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8cb53e68-3ad1-4095-8e6c-3940d330ecc0 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84281091687464990ffbca590b9343b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417e3e2f1333481795a089bef2e24bea', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:10:19.137991 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb53e68-3ad1-4095-8e6c-3940d330ecc0 service nova] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:10:19.138269 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb53e68-3ad1-4095-8e6c-3940d330ecc0 service nova] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:10:19.138555 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8cb53e68-3ad1-4095-8e6c-3940d330ecc0 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:10:19.147063 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb53e68-3ad1-4095-8e6c-3940d330ecc0 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:10:19.148662 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8cb53e68-3ad1-4095-8e6c-3940d330ecc0 service nova] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:10:19.158417 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 760/1524] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jul 13 22:10:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 22:10:19.968449 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debcce9d-a330-429d-91b3-4c1dc5dfcda8 None None] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:10:19.969341 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debcce9d-a330-429d-91b3-4c1dc5dfcda8 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:10:19.969341 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-debcce9d-a330-429d-91b3-4c1dc5dfcda8 None None] PATH_INFO: `/` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:10:19.970818 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 765/1525] 127.0.0.1 () {54 vars in 901 bytes} [Wed Jul 13 22:10:19 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 22:10:19.975820 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4960339e-b9da-4da5-a7d8-1e1fbbf0b491 None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:10:19.976243 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4960339e-b9da-4da5-a7d8-1e1fbbf0b491 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:10:19.976392 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4960339e-b9da-4da5-a7d8-1e1fbbf0b491 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:10:19.995317 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-4960339e-b9da-4da5-a7d8-1e1fbbf0b491 None None] MFA Rules not processed for user `cc99c93da5074cf5a48bc2aa9a92c5e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 22:10:19.996506 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-4960339e-b9da-4da5-a7d8-1e1fbbf0b491 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:10:20.005538 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 761/1526] 127.0.0.1 () {58 vars in 998 bytes} [Wed Jul 13 22:10:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 22:10:20.154022 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debcce9d-a330-429d-91b3-4c1dc5dfcda8 None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:10:20.179964 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-debcce9d-a330-429d-91b3-4c1dc5dfcda8 None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:10:20.190251 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18ea9d1-3694-41bd-a672-98ef0b63e623 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84281091687464990ffbca590b9343b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417e3e2f1333481795a089bef2e24bea', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:10:20.190942 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18ea9d1-3694-41bd-a672-98ef0b63e623 service nova] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:10:20.191227 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18ea9d1-3694-41bd-a672-98ef0b63e623 service nova] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:10:20.191498 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18ea9d1-3694-41bd-a672-98ef0b63e623 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:10:20.201241 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a18ea9d1-3694-41bd-a672-98ef0b63e623 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:10:20.202928 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a18ea9d1-3694-41bd-a672-98ef0b63e623 service nova] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:10:20.213256 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 766/1527] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jul 13 22:10:20 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 22:10:24.243395 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8366a2-27e4-40a9-8ae9-1ee1f5a61597 None None] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:10:24.243966 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8366a2-27e4-40a9-8ae9-1ee1f5a61597 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:10:24.243966 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a8366a2-27e4-40a9-8ae9-1ee1f5a61597 None None] PATH_INFO: `/` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:10:24.245334 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 762/1528] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 13 22:10:24 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 22:10:24.255124 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a93302-1a5c-4776-b550-c4c66cb1d19c None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:10:24.255825 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a93302-1a5c-4776-b550-c4c66cb1d19c None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:10:24.255934 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03a93302-1a5c-4776-b550-c4c66cb1d19c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:10:24.279257 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-03a93302-1a5c-4776-b550-c4c66cb1d19c None None] MFA Rules not processed for user `cc99c93da5074cf5a48bc2aa9a92c5e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 22:10:24.280396 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-03a93302-1a5c-4776-b550-c4c66cb1d19c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:10:24.289561 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 767/1529] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 13 22:10:24 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 36 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 22:10:24.368954 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8366a2-27e4-40a9-8ae9-1ee1f5a61597 None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:10:24.371866 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-9a8366a2-27e4-40a9-8ae9-1ee1f5a61597 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:10:24.470855 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a8366a2-27e4-40a9-8ae9-1ee1f5a61597 None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:10:24.475076 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e79b551-1a5c-4fec-bb37-508ae0d9dc35 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '847ec9a1bd4d4dec804318a5c845895a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417e3e2f1333481795a089bef2e24bea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:10:24.475740 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e79b551-1a5c-4fec-bb37-508ae0d9dc35 service neutron] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:10:24.476022 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e79b551-1a5c-4fec-bb37-508ae0d9dc35 service neutron] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:10:24.476292 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e79b551-1a5c-4fec-bb37-508ae0d9dc35 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:10:24.486354 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e79b551-1a5c-4fec-bb37-508ae0d9dc35 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:10:24.487827 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e79b551-1a5c-4fec-bb37-508ae0d9dc35 service neutron] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:10:24.497388 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 763/1530] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jul 13 22:10:24 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 130 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 22:10:26.501965 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce04e70-a585-4c40-b59d-389724e22765 None None] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:10:26.502925 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce04e70-a585-4c40-b59d-389724e22765 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:10:26.502925 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ce04e70-a585-4c40-b59d-389724e22765 None None] PATH_INFO: `/` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:10:26.504921 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 768/1531] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 13 22:10:26 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 22:10:26.512743 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcc19a9-4b46-444d-9675-4e4274006348 None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:10:26.513208 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcc19a9-4b46-444d-9675-4e4274006348 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:10:26.513314 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efcc19a9-4b46-444d-9675-4e4274006348 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:10:26.532674 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-efcc19a9-4b46-444d-9675-4e4274006348 None None] MFA Rules not processed for user `cc99c93da5074cf5a48bc2aa9a92c5e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 22:10:26.533873 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-efcc19a9-4b46-444d-9675-4e4274006348 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:10:26.542299 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 764/1532] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 13 22:10:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 13 22:10:26.579513 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce04e70-a585-4c40-b59d-389724e22765 None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:10:26.592492 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ce04e70-a585-4c40-b59d-389724e22765 None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:10:26.596810 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36ede237-7365-47da-8928-93f92acbf153 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '847ec9a1bd4d4dec804318a5c845895a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417e3e2f1333481795a089bef2e24bea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:10:26.597449 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ede237-7365-47da-8928-93f92acbf153 service neutron] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:10:26.597725 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ede237-7365-47da-8928-93f92acbf153 service neutron] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:10:26.597996 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36ede237-7365-47da-8928-93f92acbf153 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:10:26.607582 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ede237-7365-47da-8928-93f92acbf153 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:10:26.609016 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36ede237-7365-47da-8928-93f92acbf153 service neutron] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:10:26.618912 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 769/1533] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jul 13 22:10:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 13 22:10:30.196766 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8301547e-6ea0-478e-8a9e-a888fa3eac3c None None] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:10:30.197388 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8301547e-6ea0-478e-8a9e-a888fa3eac3c None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:10:30.197388 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8301547e-6ea0-478e-8a9e-a888fa3eac3c None None] PATH_INFO: `/` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:10:30.199110 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 765/1534] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 13 22:10:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 22:10:30.205680 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a279e61-0432-48ad-af30-419e54c2e669 None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:10:30.206349 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a279e61-0432-48ad-af30-419e54c2e669 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:10:30.206466 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a279e61-0432-48ad-af30-419e54c2e669 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:10:30.230382 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-5a279e61-0432-48ad-af30-419e54c2e669 None None] MFA Rules not processed for user `cc99c93da5074cf5a48bc2aa9a92c5e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 22:10:30.231624 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-5a279e61-0432-48ad-af30-419e54c2e669 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:10:30.240952 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 770/1535] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 13 22:10:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 22:10:30.547336 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8301547e-6ea0-478e-8a9e-a888fa3eac3c None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:10:30.550543 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-8301547e-6ea0-478e-8a9e-a888fa3eac3c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:10:30.631979 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:10:30.647003 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8301547e-6ea0-478e-8a9e-a888fa3eac3c None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:10:30.651342 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c5b0db4-5efc-43dd-a6f9-72d2dfdc4dc2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc77e1ea1ada4700b809beb1c980c5e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417e3e2f1333481795a089bef2e24bea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:10:30.651992 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5b0db4-5efc-43dd-a6f9-72d2dfdc4dc2 service cinder] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:10:30.652269 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5b0db4-5efc-43dd-a6f9-72d2dfdc4dc2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:10:30.652537 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c5b0db4-5efc-43dd-a6f9-72d2dfdc4dc2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:10:30.663037 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5b0db4-5efc-43dd-a6f9-72d2dfdc4dc2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:10:30.664475 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c5b0db4-5efc-43dd-a6f9-72d2dfdc4dc2 service cinder] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:10:30.674887 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 766/1536] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jul 13 22:10:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 22:10:32.703460 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0918318-2fad-4c03-a10d-52fc60b74496 None None] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:10:32.704510 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0918318-2fad-4c03-a10d-52fc60b74496 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:10:32.704510 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0918318-2fad-4c03-a10d-52fc60b74496 None None] PATH_INFO: `/` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:10:32.706567 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 771/1537] 127.0.0.1 () {54 vars in 916 bytes} [Wed Jul 13 22:10:32 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 22:10:32.715514 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f61aef0-9410-4304-88b8-f7b6afbee758 None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:10:32.715973 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f61aef0-9410-4304-88b8-f7b6afbee758 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:10:32.716082 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f61aef0-9410-4304-88b8-f7b6afbee758 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:10:32.736421 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-4f61aef0-9410-4304-88b8-f7b6afbee758 None None] MFA Rules not processed for user `cc99c93da5074cf5a48bc2aa9a92c5e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 22:10:32.737622 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-4f61aef0-9410-4304-88b8-f7b6afbee758 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:10:32.745832 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 767/1538] 127.0.0.1 () {58 vars in 1013 bytes} [Wed Jul 13 22:10:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 22:10:33.047996 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0918318-2fad-4c03-a10d-52fc60b74496 None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:10:33.050102 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-e0918318-2fad-4c03-a10d-52fc60b74496 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:10:33.065182 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0918318-2fad-4c03-a10d-52fc60b74496 None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:10:33.069120 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4953d054-1bba-4ae4-82e9-f917b2d2b77c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc77e1ea1ada4700b809beb1c980c5e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417e3e2f1333481795a089bef2e24bea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:10:33.069754 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4953d054-1bba-4ae4-82e9-f917b2d2b77c service cinder] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:10:33.070036 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4953d054-1bba-4ae4-82e9-f917b2d2b77c service cinder] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:10:33.070298 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4953d054-1bba-4ae4-82e9-f917b2d2b77c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:10:33.080344 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4953d054-1bba-4ae4-82e9-f917b2d2b77c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:10:33.081786 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4953d054-1bba-4ae4-82e9-f917b2d2b77c service cinder] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:10:33.092292 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 772/1539] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jul 13 22:10:33 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 22:10:42.731409 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=11 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:10:43.087670 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=33 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:12:03.235854 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd0e6cf-54a9-4147-b40f-01a644a8200a None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:12:03.236651 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd0e6cf-54a9-4147-b40f-01a644a8200a None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:12:03.236651 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dd0e6cf-54a9-4147-b40f-01a644a8200a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:12:03.263027 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-3dd0e6cf-54a9-4147-b40f-01a644a8200a None None] MFA Rules not processed for user `cc99c93da5074cf5a48bc2aa9a92c5e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 22:12:03.264264 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-3dd0e6cf-54a9-4147-b40f-01a644a8200a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:12:03.268335 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563902dd-1671-4ba6-808f-d6f94725b2fa None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:12:03.269361 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563902dd-1671-4ba6-808f-d6f94725b2fa None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:12:03.269861 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563902dd-1671-4ba6-808f-d6f94725b2fa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:12:03.281336 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 768/1540] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jul 13 22:12:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 22:12:03.290510 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30c3cf7-0c07-4ba6-928e-139452624b37 None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:12:03.290846 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30c3cf7-0c07-4ba6-928e-139452624b37 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:12:03.291075 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d30c3cf7-0c07-4ba6-928e-139452624b37 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:12:03.299573 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-563902dd-1671-4ba6-808f-d6f94725b2fa None None] MFA Rules not processed for user `cc99c93da5074cf5a48bc2aa9a92c5e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 22:12:03.301189 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-563902dd-1671-4ba6-808f-d6f94725b2fa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:12:03.318957 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 773/1541] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jul 13 22:12:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 22:12:03.320612 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b23d387-3644-4011-8e51-8feff0f4989a None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:12:03.321133 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b23d387-3644-4011-8e51-8feff0f4989a None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:12:03.321399 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-d30c3cf7-0c07-4ba6-928e-139452624b37 None None] MFA Rules not processed for user `cc99c93da5074cf5a48bc2aa9a92c5e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 22:12:03.321676 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9b23d387-3644-4011-8e51-8feff0f4989a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:12:03.323424 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-d30c3cf7-0c07-4ba6-928e-139452624b37 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:12:03.339583 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 769/1542] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jul 13 22:12:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 50 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 22:12:03.344904 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30c3cf7-0c07-4ba6-928e-139452624b37 None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:12:03.347573 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-9b23d387-3644-4011-8e51-8feff0f4989a None None] MFA Rules not processed for user `cc99c93da5074cf5a48bc2aa9a92c5e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 22:12:03.348781 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-9b23d387-3644-4011-8e51-8feff0f4989a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:12:03.362911 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d30c3cf7-0c07-4ba6-928e-139452624b37 None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:12:03.365915 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 774/1543] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jul 13 22:12:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 22:12:03.367349 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5accc364-3807-49c0-a4a0-4f34505a7e85 None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:12:03.367709 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5accc364-3807-49c0-a4a0-4f34505a7e85 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:12:03.367999 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5accc364-3807-49c0-a4a0-4f34505a7e85 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:12:03.377395 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e738c07f-a14b-4814-bd61-bf1454b58e37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:12:03.378733 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e738c07f-a14b-4814-bd61-bf1454b58e37 admin admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:12:03.379187 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e738c07f-a14b-4814-bd61-bf1454b58e37 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:12:03.379662 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e738c07f-a14b-4814-bd61-bf1454b58e37 admin admin] PATH_INFO: `/v3/domains` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:12:03.380619 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e738c07f-a14b-4814-bd61-bf1454b58e37 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 22:12:03.381084 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e738c07f-a14b-4814-bd61-bf1454b58e37 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:12:03.382893 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e738c07f-a14b-4814-bd61-bf1454b58e37 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:12:03.402513 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-5accc364-3807-49c0-a4a0-4f34505a7e85 None None] MFA Rules not processed for user `cc99c93da5074cf5a48bc2aa9a92c5e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 22:12:03.402988 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 770/1544] 10.222.0.52 () {58 vars in 1152 bytes} [Wed Jul 13 22:12:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 54 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 22:12:03.403816 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-5accc364-3807-49c0-a4a0-4f34505a7e85 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:12:03.404717 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545a1385-c4ae-4a6e-8ae3-bbbb0dcf46a2 None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:12:03.405176 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545a1385-c4ae-4a6e-8ae3-bbbb0dcf46a2 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:12:03.405618 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-545a1385-c4ae-4a6e-8ae3-bbbb0dcf46a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:12:03.423367 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 775/1545] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jul 13 22:12:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 56 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 22:12:03.423367 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5accc364-3807-49c0-a4a0-4f34505a7e85 None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:12:03.442917 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-545a1385-c4ae-4a6e-8ae3-bbbb0dcf46a2 None None] MFA Rules not processed for user `cc99c93da5074cf5a48bc2aa9a92c5e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 22:12:03.446523 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-545a1385-c4ae-4a6e-8ae3-bbbb0dcf46a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:12:03.447752 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5accc364-3807-49c0-a4a0-4f34505a7e85 None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:12:03.459984 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-196d9811-b78b-41a9-b0ec-207bada12993 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:12:03.461015 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196d9811-b78b-41a9-b0ec-207bada12993 admin admin] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:12:03.464071 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196d9811-b78b-41a9-b0ec-207bada12993 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:12:03.466658 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-196d9811-b78b-41a9-b0ec-207bada12993 admin admin] PATH_INFO: `/v3/projects` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:12:03.467297 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196d9811-b78b-41a9-b0ec-207bada12993 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:12:03.470004 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-196d9811-b78b-41a9-b0ec-207bada12993 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:12:03.488176 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 771/1546] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jul 13 22:12:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 22:12:03.489025 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545a1385-c4ae-4a6e-8ae3-bbbb0dcf46a2 None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:12:03.508868 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-545a1385-c4ae-4a6e-8ae3-bbbb0dcf46a2 None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:12:03.515795 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2b58bf-0483-4bfe-88ee-94af074691fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:12:03.516481 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2b58bf-0483-4bfe-88ee-94af074691fc admin admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:12:03.518693 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2b58bf-0483-4bfe-88ee-94af074691fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:12:03.518996 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af2b58bf-0483-4bfe-88ee-94af074691fc admin admin] PATH_INFO: `/v3/domains` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:12:03.522072 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2b58bf-0483-4bfe-88ee-94af074691fc admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 22:12:03.522399 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2b58bf-0483-4bfe-88ee-94af074691fc admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:12:03.524064 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af2b58bf-0483-4bfe-88ee-94af074691fc admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:12:03.530175 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 776/1547] 10.222.0.52 () {60 vars in 1152 bytes} [Wed Jul 13 22:12:03 2022] POST /identity/v3/projects => generated 385 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 22:12:03.531721 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad168da-e309-4207-821a-14a23672f5be None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:12:03.532011 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad168da-e309-4207-821a-14a23672f5be None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:12:03.532287 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dad168da-e309-4207-821a-14a23672f5be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:12:03.557874 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 772/1548] 10.222.0.52 () {58 vars in 1152 bytes} [Wed Jul 13 22:12:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 22:12:03.558076 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-dad168da-e309-4207-821a-14a23672f5be None None] MFA Rules not processed for user `cc99c93da5074cf5a48bc2aa9a92c5e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 22:12:03.559269 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2b58bf-0483-4bfe-88ee-94af074691fc admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:12:03.559402 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-dad168da-e309-4207-821a-14a23672f5be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:12:03.574175 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af2b58bf-0483-4bfe-88ee-94af074691fc admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:12:03.639707 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 777/1549] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jul 13 22:12:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 22:12:03.641347 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cd976d-f430-453c-9d46-2865b5dfe9b8 None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:12:03.641864 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cd976d-f430-453c-9d46-2865b5dfe9b8 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:12:03.642440 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66cd976d-f430-453c-9d46-2865b5dfe9b8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:12:03.654572 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac7f4c5-483a-47ba-ac1b-1379e9234040 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:12:03.655671 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac7f4c5-483a-47ba-ac1b-1379e9234040 admin admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:12:03.656088 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac7f4c5-483a-47ba-ac1b-1379e9234040 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:12:03.656519 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ac7f4c5-483a-47ba-ac1b-1379e9234040 admin admin] PATH_INFO: `/v3/domains` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:12:03.657566 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac7f4c5-483a-47ba-ac1b-1379e9234040 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 22:12:03.657877 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac7f4c5-483a-47ba-ac1b-1379e9234040 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:12:03.659652 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ac7f4c5-483a-47ba-ac1b-1379e9234040 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:12:03.660871 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-66cd976d-f430-453c-9d46-2865b5dfe9b8 None None] MFA Rules not processed for user `cc99c93da5074cf5a48bc2aa9a92c5e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 22:12:03.662927 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-66cd976d-f430-453c-9d46-2865b5dfe9b8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:12:03.674691 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 778/1550] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jul 13 22:12:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 22:12:03.674691 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dedeed9-045b-4bda-8900-676a5cbf6a68 None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:12:03.674691 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dedeed9-045b-4bda-8900-676a5cbf6a68 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:12:03.674691 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dedeed9-045b-4bda-8900-676a5cbf6a68 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:12:03.682802 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 773/1551] 10.222.0.52 () {58 vars in 1152 bytes} [Wed Jul 13 22:12:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 119 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 22:12:03.682802 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac7f4c5-483a-47ba-ac1b-1379e9234040 admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:12:03.690307 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ac7f4c5-483a-47ba-ac1b-1379e9234040 admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:12:03.695406 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0624428c-8151-41ed-9c2a-a93c6bbd5b23 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:12:03.696160 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0624428c-8151-41ed-9c2a-a93c6bbd5b23 admin admin] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:12:03.696505 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0624428c-8151-41ed-9c2a-a93c6bbd5b23 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:12:03.696850 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0624428c-8151-41ed-9c2a-a93c6bbd5b23 admin admin] PATH_INFO: `/v3/users` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:12:03.697183 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-7dedeed9-045b-4bda-8900-676a5cbf6a68 None None] MFA Rules not processed for user `cc99c93da5074cf5a48bc2aa9a92c5e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 22:12:03.698155 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0624428c-8151-41ed-9c2a-a93c6bbd5b23 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:12:03.698903 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-7dedeed9-045b-4bda-8900-676a5cbf6a68 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:12:03.700270 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0624428c-8151-41ed-9c2a-a93c6bbd5b23 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:12:03.707510 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 779/1552] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jul 13 22:12:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 22:12:03.708408 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dedeed9-045b-4bda-8900-676a5cbf6a68 None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:12:03.719218 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dedeed9-045b-4bda-8900-676a5cbf6a68 None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:12:03.724961 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5a98647-667f-4218-9c6b-2bddbb3a1222 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:12:03.724961 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a98647-667f-4218-9c6b-2bddbb3a1222 admin admin] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:12:03.725236 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a98647-667f-4218-9c6b-2bddbb3a1222 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:12:03.725414 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5a98647-667f-4218-9c6b-2bddbb3a1222 admin admin] PATH_INFO: `/v3/projects` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:12:03.726148 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a98647-667f-4218-9c6b-2bddbb3a1222 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:12:03.728100 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5a98647-667f-4218-9c6b-2bddbb3a1222 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:12:03.733413 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 774/1553] 10.222.0.52 () {60 vars in 1143 bytes} [Wed Jul 13 22:12:03 2022] POST /identity/v3/users => generated 293 bytes in 55 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 22:12:03.734736 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e20781-e98b-46d6-8657-ff841dbab8b7 None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:12:03.735026 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e20781-e98b-46d6-8657-ff841dbab8b7 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:12:03.735300 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e20781-e98b-46d6-8657-ff841dbab8b7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:12:03.756118 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-38e20781-e98b-46d6-8657-ff841dbab8b7 None None] MFA Rules not processed for user `cc99c93da5074cf5a48bc2aa9a92c5e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 22:12:03.759648 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 780/1554] 10.222.0.52 () {60 vars in 1152 bytes} [Wed Jul 13 22:12:03 2022] POST /identity/v3/projects => generated 375 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 22:12:03.759787 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-38e20781-e98b-46d6-8657-ff841dbab8b7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:12:03.759890 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc482f9-35f7-47b4-b9e2-1e60fc1d38af None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:12:03.759890 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc482f9-35f7-47b4-b9e2-1e60fc1d38af None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:12:03.759890 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9dc482f9-35f7-47b4-b9e2-1e60fc1d38af None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:12:03.775373 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-9dc482f9-35f7-47b4-b9e2-1e60fc1d38af None None] MFA Rules not processed for user `cc99c93da5074cf5a48bc2aa9a92c5e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 22:12:03.776554 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-9dc482f9-35f7-47b4-b9e2-1e60fc1d38af None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:12:03.818233 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 775/1555] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jul 13 22:12:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 22:12:03.819183 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e20781-e98b-46d6-8657-ff841dbab8b7 None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:12:03.830812 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e20781-e98b-46d6-8657-ff841dbab8b7 None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:12:03.835740 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-91a0513b-34cf-49ba-ab8f-686291e06a36 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:12:03.836396 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a0513b-34cf-49ba-ab8f-686291e06a36 admin admin] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:12:03.836673 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a0513b-34cf-49ba-ab8f-686291e06a36 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:12:03.836945 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-91a0513b-34cf-49ba-ab8f-686291e06a36 admin admin] PATH_INFO: `/v3/projects` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:12:03.837634 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a0513b-34cf-49ba-ab8f-686291e06a36 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:12:03.838350 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 781/1556] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jul 13 22:12:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 22:12:03.838681 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: WARNING py.warnings [None req-91a0513b-34cf-49ba-ab8f-686291e06a36 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 13 22:12:03.838681 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: warnings.warn(msg) Jul 13 22:12:03.838681 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: Jul 13 22:12:03.839708 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b117a6-57f0-4544-9615-fca3284a7b39 None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:12:03.839858 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-91a0513b-34cf-49ba-ab8f-686291e06a36 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:12:03.840044 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b117a6-57f0-4544-9615-fca3284a7b39 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:12:03.840265 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0b117a6-57f0-4544-9615-fca3284a7b39 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:12:03.858214 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-d0b117a6-57f0-4544-9615-fca3284a7b39 None None] MFA Rules not processed for user `cc99c93da5074cf5a48bc2aa9a92c5e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 22:12:03.859734 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-d0b117a6-57f0-4544-9615-fca3284a7b39 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:12:03.868017 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 782/1557] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jul 13 22:12:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 22:12:03.869219 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956ffd24-3d4f-4050-a06f-c119449cef7c None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:12:03.869502 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956ffd24-3d4f-4050-a06f-c119449cef7c None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:12:03.869679 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 776/1558] 10.222.0.52 () {60 vars in 1152 bytes} [Wed Jul 13 22:12:03 2022] POST /identity/v3/projects => generated 377 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 22:12:03.869793 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-956ffd24-3d4f-4050-a06f-c119449cef7c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:12:03.871012 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a47423-e8bf-47cd-b3c1-46aa841f0198 None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:12:03.871353 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a47423-e8bf-47cd-b3c1-46aa841f0198 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:12:03.871661 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23a47423-e8bf-47cd-b3c1-46aa841f0198 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:12:03.888432 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-956ffd24-3d4f-4050-a06f-c119449cef7c None None] MFA Rules not processed for user `cc99c93da5074cf5a48bc2aa9a92c5e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 22:12:03.889666 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-956ffd24-3d4f-4050-a06f-c119449cef7c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:12:03.891475 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-23a47423-e8bf-47cd-b3c1-46aa841f0198 None None] MFA Rules not processed for user `cc99c93da5074cf5a48bc2aa9a92c5e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 22:12:03.892658 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-23a47423-e8bf-47cd-b3c1-46aa841f0198 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:12:03.894364 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:12:03.897318 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:12:03.954781 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 777/1559] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jul 13 22:12:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 22:12:03.955042 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 783/1560] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jul 13 22:12:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 22:12:03.955890 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a47423-e8bf-47cd-b3c1-46aa841f0198 None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:12:03.956022 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956ffd24-3d4f-4050-a06f-c119449cef7c None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:12:03.965809 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23a47423-e8bf-47cd-b3c1-46aa841f0198 None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:12:03.968005 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-956ffd24-3d4f-4050-a06f-c119449cef7c None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:12:03.970797 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-961b9750-4ee0-4b8c-8fc9-4938caab88fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:12:03.971580 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961b9750-4ee0-4b8c-8fc9-4938caab88fc admin admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:12:03.971949 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961b9750-4ee0-4b8c-8fc9-4938caab88fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:12:03.972290 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-961b9750-4ee0-4b8c-8fc9-4938caab88fc admin admin] PATH_INFO: `/v3/roles` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:12:03.972608 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb4cc5b-fb11-4491-8bc3-a8a0f90e6130 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:12:03.973167 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-961b9750-4ee0-4b8c-8fc9-4938caab88fc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:12:03.973343 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb4cc5b-fb11-4491-8bc3-a8a0f90e6130 admin admin] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:12:03.973579 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb4cc5b-fb11-4491-8bc3-a8a0f90e6130 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:12:03.973908 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cb4cc5b-fb11-4491-8bc3-a8a0f90e6130 admin admin] PATH_INFO: `/v3/users` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:12:03.974806 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb4cc5b-fb11-4491-8bc3-a8a0f90e6130 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:12:03.975457 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-961b9750-4ee0-4b8c-8fc9-4938caab88fc admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:12:03.975834 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: WARNING py.warnings [None req-2cb4cc5b-fb11-4491-8bc3-a8a0f90e6130 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 13 22:12:03.975834 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: warnings.warn(msg) Jul 13 22:12:03.975834 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: Jul 13 22:12:03.977024 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cb4cc5b-fb11-4491-8bc3-a8a0f90e6130 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:12:03.987142 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 778/1561] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jul 13 22:12:03 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 22:12:03.990633 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e849824-57e7-4b41-b4ce-38aae60a47e0 None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:12:03.990633 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e849824-57e7-4b41-b4ce-38aae60a47e0 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:12:03.990633 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1e849824-57e7-4b41-b4ce-38aae60a47e0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:12:04.003006 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 784/1562] 10.222.0.52 () {60 vars in 1143 bytes} [Wed Jul 13 22:12:03 2022] POST /identity/v3/users => generated 288 bytes in 47 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 22:12:04.003954 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb4cc5b-fb11-4491-8bc3-a8a0f90e6130 admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:12:04.012887 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-1e849824-57e7-4b41-b4ce-38aae60a47e0 None None] MFA Rules not processed for user `cc99c93da5074cf5a48bc2aa9a92c5e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 22:12:04.013993 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-1e849824-57e7-4b41-b4ce-38aae60a47e0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:12:04.014140 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cb4cc5b-fb11-4491-8bc3-a8a0f90e6130 admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:12:04.018853 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86d59f1-ccb6-46ba-ac58-263fdaa077cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:12:04.019509 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86d59f1-ccb6-46ba-ac58-263fdaa077cb admin admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:12:04.019794 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86d59f1-ccb6-46ba-ac58-263fdaa077cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:12:04.020070 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d86d59f1-ccb6-46ba-ac58-263fdaa077cb admin admin] PATH_INFO: `/v3/domains` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:12:04.020860 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d86d59f1-ccb6-46ba-ac58-263fdaa077cb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 22:12:04.021163 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d86d59f1-ccb6-46ba-ac58-263fdaa077cb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:12:04.022146 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 779/1563] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jul 13 22:12:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 22:12:04.022260 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: WARNING py.warnings [None req-d86d59f1-ccb6-46ba-ac58-263fdaa077cb admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:list_domains": "role:reader and system_scope:all" failed scope check. The token used to make the request was project scoped but the policy requires ['system'] scope. This behavior may change in the future where using the intended scope is required Jul 13 22:12:04.022260 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: warnings.warn(msg) Jul 13 22:12:04.022260 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: Jul 13 22:12:04.022924 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e849824-57e7-4b41-b4ce-38aae60a47e0 None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:12:04.023083 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d86d59f1-ccb6-46ba-ac58-263fdaa077cb admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:12:04.032341 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1e849824-57e7-4b41-b4ce-38aae60a47e0 None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:12:04.043833 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 785/1564] 10.222.0.52 () {58 vars in 1152 bytes} [Wed Jul 13 22:12:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 22:12:04.043833 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86d59f1-ccb6-46ba-ac58-263fdaa077cb admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:12:04.047239 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2aa3c11-c5b3-47c5-a9e8-ae3b102b95fa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:12:04.047922 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2aa3c11-c5b3-47c5-a9e8-ae3b102b95fa admin admin] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:12:04.048454 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2aa3c11-c5b3-47c5-a9e8-ae3b102b95fa admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:12:04.048732 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2aa3c11-c5b3-47c5-a9e8-ae3b102b95fa admin admin] PATH_INFO: `/v3/users` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:12:04.049438 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2aa3c11-c5b3-47c5-a9e8-ae3b102b95fa admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:12:04.051377 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2aa3c11-c5b3-47c5-a9e8-ae3b102b95fa admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:12:04.053399 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d86d59f1-ccb6-46ba-ac58-263fdaa077cb admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:12:04.058072 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11347bc4-a100-42b4-b904-be51e5d0aee6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:12:04.058786 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11347bc4-a100-42b4-b904-be51e5d0aee6 admin admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:12:04.059065 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11347bc4-a100-42b4-b904-be51e5d0aee6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:12:04.059313 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11347bc4-a100-42b4-b904-be51e5d0aee6 admin admin] PATH_INFO: `/v3/domains` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:12:04.060064 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11347bc4-a100-42b4-b904-be51e5d0aee6 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 22:12:04.060485 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11347bc4-a100-42b4-b904-be51e5d0aee6 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:12:04.061979 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11347bc4-a100-42b4-b904-be51e5d0aee6 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:12:04.076279 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 780/1565] 10.222.0.52 () {60 vars in 1143 bytes} [Wed Jul 13 22:12:04 2022] POST /identity/v3/users => generated 289 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 22:12:04.076507 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 786/1566] 10.222.0.52 () {58 vars in 1152 bytes} [Wed Jul 13 22:12:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 22:12:04.076636 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f3d705-07b1-4c5e-9691-304b38653d0f None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:12:04.076636 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f3d705-07b1-4c5e-9691-304b38653d0f None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:12:04.076820 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11347bc4-a100-42b4-b904-be51e5d0aee6 admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:12:04.076938 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7f3d705-07b1-4c5e-9691-304b38653d0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:12:04.090831 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11347bc4-a100-42b4-b904-be51e5d0aee6 admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:12:04.095372 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c39193-5aa1-43fd-901e-bfe728b30d10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:12:04.096065 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c39193-5aa1-43fd-901e-bfe728b30d10 admin admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:12:04.096346 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c39193-5aa1-43fd-901e-bfe728b30d10 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:12:04.096442 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-f7f3d705-07b1-4c5e-9691-304b38653d0f None None] MFA Rules not processed for user `cc99c93da5074cf5a48bc2aa9a92c5e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 22:12:04.096621 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15c39193-5aa1-43fd-901e-bfe728b30d10 admin admin] PATH_INFO: `/v3/domains` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:12:04.097376 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c39193-5aa1-43fd-901e-bfe728b30d10 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 22:12:04.097645 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-f7f3d705-07b1-4c5e-9691-304b38653d0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:12:04.097756 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c39193-5aa1-43fd-901e-bfe728b30d10 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:12:04.099355 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-15c39193-5aa1-43fd-901e-bfe728b30d10 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:12:04.107216 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 781/1567] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jul 13 22:12:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 22:12:04.108080 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f3d705-07b1-4c5e-9691-304b38653d0f None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:12:04.118471 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 787/1568] 10.222.0.52 () {58 vars in 1152 bytes} [Wed Jul 13 22:12:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 22:12:04.118471 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c39193-5aa1-43fd-901e-bfe728b30d10 admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:12:04.119033 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7f3d705-07b1-4c5e-9691-304b38653d0f None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:12:04.123979 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e824256d-3b54-4d43-a411-8f66eba6ab04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:12:04.124671 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e824256d-3b54-4d43-a411-8f66eba6ab04 admin admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:12:04.124954 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e824256d-3b54-4d43-a411-8f66eba6ab04 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:12:04.125226 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e824256d-3b54-4d43-a411-8f66eba6ab04 admin admin] PATH_INFO: `/v3/roles` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:12:04.125963 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e824256d-3b54-4d43-a411-8f66eba6ab04 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:12:04.127622 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e824256d-3b54-4d43-a411-8f66eba6ab04 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:12:04.128186 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15c39193-5aa1-43fd-901e-bfe728b30d10 admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:12:04.133672 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818e33b3-9a44-4b02-9c88-f35457edacdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:12:04.134362 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818e33b3-9a44-4b02-9c88-f35457edacdd admin admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:12:04.134678 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818e33b3-9a44-4b02-9c88-f35457edacdd admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:12:04.134954 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-818e33b3-9a44-4b02-9c88-f35457edacdd admin admin] PATH_INFO: `/v3/roles` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:12:04.135680 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-818e33b3-9a44-4b02-9c88-f35457edacdd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:12:04.137273 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-818e33b3-9a44-4b02-9c88-f35457edacdd admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:12:04.138685 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 782/1569] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jul 13 22:12:04 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 22:12:04.142525 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e824256d-3b54-4d43-a411-8f66eba6ab04 admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:12:04.148368 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 788/1570] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jul 13 22:12:04 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 22:12:04.149240 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818e33b3-9a44-4b02-9c88-f35457edacdd admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:12:04.153294 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e824256d-3b54-4d43-a411-8f66eba6ab04 admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:12:04.158683 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-818e33b3-9a44-4b02-9c88-f35457edacdd admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:12:04.158843 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a346b0c8-e945-4310-b239-a41d87c284de admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:12:04.159502 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a346b0c8-e945-4310-b239-a41d87c284de admin admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:12:04.159772 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a346b0c8-e945-4310-b239-a41d87c284de admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:12:04.160098 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a346b0c8-e945-4310-b239-a41d87c284de admin admin] PATH_INFO: `/v3/domains` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:12:04.160766 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a346b0c8-e945-4310-b239-a41d87c284de admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 22:12:04.161069 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a346b0c8-e945-4310-b239-a41d87c284de admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:12:04.162682 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a346b0c8-e945-4310-b239-a41d87c284de admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:12:04.164001 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94084e6-c9e5-4e7e-afad-e86a3bb5b69a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:12:04.164637 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94084e6-c9e5-4e7e-afad-e86a3bb5b69a admin admin] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:12:04.164916 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94084e6-c9e5-4e7e-afad-e86a3bb5b69a admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:12:04.165195 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a94084e6-c9e5-4e7e-afad-e86a3bb5b69a admin admin] PATH_INFO: `/v3/projects` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:12:04.165874 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a94084e6-c9e5-4e7e-afad-e86a3bb5b69a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:12:04.167772 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a94084e6-c9e5-4e7e-afad-e86a3bb5b69a admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:12:04.175895 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 783/1571] 10.222.0.52 () {58 vars in 1152 bytes} [Wed Jul 13 22:12:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 22:12:04.176797 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a346b0c8-e945-4310-b239-a41d87c284de admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:12:04.187863 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a346b0c8-e945-4310-b239-a41d87c284de admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:12:04.194391 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6aea87-2b6e-45fd-95b4-f66ab3328557 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:12:04.195080 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6aea87-2b6e-45fd-95b4-f66ab3328557 admin admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:12:04.195359 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6aea87-2b6e-45fd-95b4-f66ab3328557 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:12:04.195634 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d6aea87-2b6e-45fd-95b4-f66ab3328557 admin admin] PATH_INFO: `/v3/roles` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:12:04.196356 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6aea87-2b6e-45fd-95b4-f66ab3328557 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:12:04.200722 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 789/1572] 10.222.0.52 () {60 vars in 1152 bytes} [Wed Jul 13 22:12:04 2022] POST /identity/v3/projects => generated 393 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 22:12:04.200722 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94084e6-c9e5-4e7e-afad-e86a3bb5b69a admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:12:04.200978 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d6aea87-2b6e-45fd-95b4-f66ab3328557 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:12:04.209742 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 784/1573] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jul 13 22:12:04 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 22:12:04.213641 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6aea87-2b6e-45fd-95b4-f66ab3328557 admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:12:04.223483 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d6aea87-2b6e-45fd-95b4-f66ab3328557 admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:12:04.261117 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a94084e6-c9e5-4e7e-afad-e86a3bb5b69a admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:12:04.266002 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3efd8e91-9ef8-48cc-8182-40e59cdf8062 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:12:04.266653 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efd8e91-9ef8-48cc-8182-40e59cdf8062 admin admin] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:12:04.266934 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efd8e91-9ef8-48cc-8182-40e59cdf8062 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:12:04.267210 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3efd8e91-9ef8-48cc-8182-40e59cdf8062 admin admin] PATH_INFO: `/v3/projects` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:12:04.267910 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3efd8e91-9ef8-48cc-8182-40e59cdf8062 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:12:04.269842 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3efd8e91-9ef8-48cc-8182-40e59cdf8062 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:12:04.283403 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf9fb3b-24e9-46ed-b29e-d9faed075413 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:12:04.284304 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf9fb3b-24e9-46ed-b29e-d9faed075413 admin admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:12:04.284584 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf9fb3b-24e9-46ed-b29e-d9faed075413 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:12:04.285005 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-daf9fb3b-24e9-46ed-b29e-d9faed075413 admin admin] PATH_INFO: `/v3/domains` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:12:04.285686 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf9fb3b-24e9-46ed-b29e-d9faed075413 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 22:12:04.286205 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf9fb3b-24e9-46ed-b29e-d9faed075413 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:12:04.288531 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-daf9fb3b-24e9-46ed-b29e-d9faed075413 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:12:04.301130 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 790/1574] 10.222.0.52 () {60 vars in 1152 bytes} [Wed Jul 13 22:12:04 2022] POST /identity/v3/projects => generated 385 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 22:12:04.301130 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3efd8e91-9ef8-48cc-8182-40e59cdf8062 admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:12:04.301960 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 785/1575] 10.222.0.52 () {58 vars in 1152 bytes} [Wed Jul 13 22:12:04 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 89 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 22:12:04.303377 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf9fb3b-24e9-46ed-b29e-d9faed075413 admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:12:04.305776 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:12:04.319438 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-daf9fb3b-24e9-46ed-b29e-d9faed075413 admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:12:04.344968 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:12:04.364645 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3efd8e91-9ef8-48cc-8182-40e59cdf8062 admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:12:04.370223 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7f52ac-77d6-404d-9724-182e365cd228 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:12:04.370929 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7f52ac-77d6-404d-9724-182e365cd228 admin admin] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:12:04.371217 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7f52ac-77d6-404d-9724-182e365cd228 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:12:04.371502 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a7f52ac-77d6-404d-9724-182e365cd228 admin admin] PATH_INFO: `/v3/projects` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:12:04.372224 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a7f52ac-77d6-404d-9724-182e365cd228 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:12:04.374303 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8a7f52ac-77d6-404d-9724-182e365cd228 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:12:04.377113 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a1ca9a-82af-44b5-b62b-c8c2b76be438 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:12:04.378017 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a1ca9a-82af-44b5-b62b-c8c2b76be438 admin admin] REQUEST_METHOD: `PUT` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:12:04.378558 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a1ca9a-82af-44b5-b62b-c8c2b76be438 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:12:04.378670 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43a1ca9a-82af-44b5-b62b-c8c2b76be438 admin admin] PATH_INFO: `/v3/projects/147ec3e62412490a9a8768973960f323/users/4ca2967918434dab95ed42fac6bbd318/roles/2a5e86976394498abb6310697cdb3f8b` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:12:04.393224 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a1ca9a-82af-44b5-b62b-c8c2b76be438 admin admin] RBAC: Authorizing `identity:create_grant(project_id=147ec3e62412490a9a8768973960f323, user_id=4ca2967918434dab95ed42fac6bbd318, role_id=2a5e86976394498abb6310697cdb3f8b)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:12:04.394340 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: WARNING py.warnings [None req-43a1ca9a-82af-44b5-b62b-c8c2b76be438 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:create_grant": "(role:admin and system_scope:all) or ((role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.user.domain_id)s and domain_id:%(target.domain.id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.project.domain_id)s) or (role:admin and domain_id:%(target.group.domain_id)s and domain_id:%(target.domain.id)s)) and (domain_id:%(target.role.domain_id)s or None:%(target.role.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 13 22:12:04.394340 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: warnings.warn(msg) Jul 13 22:12:04.394340 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: Jul 13 22:12:04.396292 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43a1ca9a-82af-44b5-b62b-c8c2b76be438 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:12:04.402442 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 791/1576] 10.222.0.52 () {60 vars in 1152 bytes} [Wed Jul 13 22:12:04 2022] POST /identity/v3/projects => generated 367 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 22:12:04.403550 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7f52ac-77d6-404d-9724-182e365cd228 admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:12:04.410794 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 786/1577] 10.222.0.52 () {60 vars in 1482 bytes} [Wed Jul 13 22:12:04 2022] PUT /identity/v3/projects/147ec3e62412490a9a8768973960f323/users/4ca2967918434dab95ed42fac6bbd318/roles/2a5e86976394498abb6310697cdb3f8b => generated 0 bytes in 108 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 22:12:04.411697 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a1ca9a-82af-44b5-b62b-c8c2b76be438 admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:12:04.466173 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a7f52ac-77d6-404d-9724-182e365cd228 admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:12:04.483891 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43a1ca9a-82af-44b5-b62b-c8c2b76be438 admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:12:04.489662 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21ff708-b68d-4290-b982-02077e431779 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:12:04.490412 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21ff708-b68d-4290-b982-02077e431779 admin admin] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:12:04.491195 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21ff708-b68d-4290-b982-02077e431779 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:12:04.491383 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a21ff708-b68d-4290-b982-02077e431779 admin admin] PATH_INFO: `/v3/projects` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:12:04.492074 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21ff708-b68d-4290-b982-02077e431779 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:12:04.494394 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a21ff708-b68d-4290-b982-02077e431779 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:12:04.522095 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 787/1578] 10.222.0.52 () {60 vars in 1152 bytes} [Wed Jul 13 22:12:04 2022] POST /identity/v3/projects => generated 375 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 22:12:04.522928 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21ff708-b68d-4290-b982-02077e431779 admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:12:04.525123 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778ee691-69a6-4592-91f0-1d8d0dd9dfc2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:12:04.525920 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778ee691-69a6-4592-91f0-1d8d0dd9dfc2 admin admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:12:04.526292 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778ee691-69a6-4592-91f0-1d8d0dd9dfc2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:12:04.526689 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-778ee691-69a6-4592-91f0-1d8d0dd9dfc2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:12:04.527519 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-778ee691-69a6-4592-91f0-1d8d0dd9dfc2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:12:04.529329 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-778ee691-69a6-4592-91f0-1d8d0dd9dfc2 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:12:04.539649 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 792/1579] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jul 13 22:12:04 2022] GET /identity/v3/roles => generated 1420 bytes in 137 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 22:12:04.540649 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778ee691-69a6-4592-91f0-1d8d0dd9dfc2 admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:12:04.550328 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-778ee691-69a6-4592-91f0-1d8d0dd9dfc2 admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:12:04.581797 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:12:04.593000 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a21ff708-b68d-4290-b982-02077e431779 admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:12:04.597704 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9202917-5b5d-4f89-a3f1-972d127e1f59 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:12:04.598386 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9202917-5b5d-4f89-a3f1-972d127e1f59 admin admin] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:12:04.598690 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9202917-5b5d-4f89-a3f1-972d127e1f59 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:12:04.599059 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f9202917-5b5d-4f89-a3f1-972d127e1f59 admin admin] PATH_INFO: `/v3/users` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:12:04.599756 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9202917-5b5d-4f89-a3f1-972d127e1f59 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:12:04.601685 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f9202917-5b5d-4f89-a3f1-972d127e1f59 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:12:04.613825 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b529b6-9ab1-4a04-aedb-fe7ed76fa431 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:12:04.614606 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b529b6-9ab1-4a04-aedb-fe7ed76fa431 admin admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:12:04.614900 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b529b6-9ab1-4a04-aedb-fe7ed76fa431 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:12:04.615159 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77b529b6-9ab1-4a04-aedb-fe7ed76fa431 admin admin] PATH_INFO: `/v3/roles` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:12:04.615950 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b529b6-9ab1-4a04-aedb-fe7ed76fa431 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:12:04.617646 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77b529b6-9ab1-4a04-aedb-fe7ed76fa431 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:12:04.627357 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 788/1580] 10.222.0.52 () {60 vars in 1143 bytes} [Wed Jul 13 22:12:04 2022] POST /identity/v3/users => generated 297 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 22:12:04.627357 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9202917-5b5d-4f89-a3f1-972d127e1f59 admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:12:04.634203 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 793/1581] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jul 13 22:12:04 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 22:12:04.634203 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b529b6-9ab1-4a04-aedb-fe7ed76fa431 admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:12:04.635243 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f9202917-5b5d-4f89-a3f1-972d127e1f59 admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:12:04.640319 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8f1b84-6a19-4c37-9d51-e05706ef0f7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:12:04.641019 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8f1b84-6a19-4c37-9d51-e05706ef0f7e admin admin] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:12:04.641304 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8f1b84-6a19-4c37-9d51-e05706ef0f7e admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:12:04.641579 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f8f1b84-6a19-4c37-9d51-e05706ef0f7e admin admin] PATH_INFO: `/v3/users` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:12:04.642277 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f8f1b84-6a19-4c37-9d51-e05706ef0f7e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:12:04.644027 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77b529b6-9ab1-4a04-aedb-fe7ed76fa431 admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:12:04.644245 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1f8f1b84-6a19-4c37-9d51-e05706ef0f7e admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:12:04.648878 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9f6517c-3a4a-435d-b492-be7ca5e4ec03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:12:04.649517 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f6517c-3a4a-435d-b492-be7ca5e4ec03 admin admin] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:12:04.649793 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f6517c-3a4a-435d-b492-be7ca5e4ec03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:12:04.650060 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9f6517c-3a4a-435d-b492-be7ca5e4ec03 admin admin] PATH_INFO: `/v3/projects` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:12:04.650770 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f6517c-3a4a-435d-b492-be7ca5e4ec03 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:12:04.652648 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9f6517c-3a4a-435d-b492-be7ca5e4ec03 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:12:04.665797 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 789/1582] 10.222.0.52 () {60 vars in 1143 bytes} [Wed Jul 13 22:12:04 2022] POST /identity/v3/users => generated 293 bytes in 41 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 22:12:04.666723 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8f1b84-6a19-4c37-9d51-e05706ef0f7e admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:12:04.677845 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f8f1b84-6a19-4c37-9d51-e05706ef0f7e admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:12:04.681487 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 794/1583] 10.222.0.52 () {60 vars in 1152 bytes} [Wed Jul 13 22:12:04 2022] POST /identity/v3/projects => generated 387 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 22:12:04.682722 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34cd623-10c1-4b2b-a260-e19eb7c0f85d None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:12:04.683011 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34cd623-10c1-4b2b-a260-e19eb7c0f85d None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:12:04.683289 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f34cd623-10c1-4b2b-a260-e19eb7c0f85d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:12:04.686117 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0521e56f-bcb8-4f77-bce2-d78147398808 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:12:04.686905 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0521e56f-bcb8-4f77-bce2-d78147398808 admin admin] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:12:04.687241 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0521e56f-bcb8-4f77-bce2-d78147398808 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:12:04.687573 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0521e56f-bcb8-4f77-bce2-d78147398808 admin admin] PATH_INFO: `/v3/users` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:12:04.688334 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0521e56f-bcb8-4f77-bce2-d78147398808 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:12:04.691538 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0521e56f-bcb8-4f77-bce2-d78147398808 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:12:04.698860 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:12:04.712214 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-f34cd623-10c1-4b2b-a260-e19eb7c0f85d None None] MFA Rules not processed for user `4ca2967918434dab95ed42fac6bbd318`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 22:12:04.713691 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-f34cd623-10c1-4b2b-a260-e19eb7c0f85d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:12:04.719956 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 790/1584] 10.222.0.52 () {60 vars in 1143 bytes} [Wed Jul 13 22:12:04 2022] POST /identity/v3/users => generated 284 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 22:12:04.719956 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0521e56f-bcb8-4f77-bce2-d78147398808 admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:12:04.729475 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0521e56f-bcb8-4f77-bce2-d78147398808 admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:12:04.780396 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 795/1585] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jul 13 22:12:04 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 22:12:04.781128 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34cd623-10c1-4b2b-a260-e19eb7c0f85d None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:12:04.784981 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470bb8d3-9c2d-464d-8479-6d54a352c27d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:12:04.785764 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470bb8d3-9c2d-464d-8479-6d54a352c27d admin admin] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:12:04.786142 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470bb8d3-9c2d-464d-8479-6d54a352c27d admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:12:04.786577 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470bb8d3-9c2d-464d-8479-6d54a352c27d admin admin] PATH_INFO: `/v3/users` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:12:04.787478 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470bb8d3-9c2d-464d-8479-6d54a352c27d admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:12:04.789506 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470bb8d3-9c2d-464d-8479-6d54a352c27d admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:12:04.791771 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f34cd623-10c1-4b2b-a260-e19eb7c0f85d None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:12:04.796372 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188397a9-9dac-41dc-bdb8-6eb37a1f4feb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:12:04.797255 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188397a9-9dac-41dc-bdb8-6eb37a1f4feb admin admin] REQUEST_METHOD: `PUT` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:12:04.797526 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188397a9-9dac-41dc-bdb8-6eb37a1f4feb admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:12:04.797791 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-188397a9-9dac-41dc-bdb8-6eb37a1f4feb admin admin] PATH_INFO: `/v3/projects/ed0c80a1d63b412e9936acf0dca3b2ec/users/18b51e7024fc46a8af3644af27ab886f/roles/2a5e86976394498abb6310697cdb3f8b` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:12:04.814320 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 791/1586] 10.222.0.52 () {60 vars in 1143 bytes} [Wed Jul 13 22:12:04 2022] POST /identity/v3/users => generated 288 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 22:12:04.815044 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470bb8d3-9c2d-464d-8479-6d54a352c27d admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:12:04.817566 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188397a9-9dac-41dc-bdb8-6eb37a1f4feb admin admin] RBAC: Authorizing `identity:create_grant(project_id=ed0c80a1d63b412e9936acf0dca3b2ec, user_id=18b51e7024fc46a8af3644af27ab886f, role_id=2a5e86976394498abb6310697cdb3f8b)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:12:04.820382 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-188397a9-9dac-41dc-bdb8-6eb37a1f4feb admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:12:04.825921 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470bb8d3-9c2d-464d-8479-6d54a352c27d admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:12:04.831337 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceaec259-0416-4ca1-b4f0-457ac4f04efb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:12:04.832368 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceaec259-0416-4ca1-b4f0-457ac4f04efb admin admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:12:04.832658 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceaec259-0416-4ca1-b4f0-457ac4f04efb admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:12:04.832919 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ceaec259-0416-4ca1-b4f0-457ac4f04efb admin admin] PATH_INFO: `/v3/roles` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:12:04.834013 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceaec259-0416-4ca1-b4f0-457ac4f04efb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:12:04.835650 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ceaec259-0416-4ca1-b4f0-457ac4f04efb admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:12:04.836392 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 796/1587] 10.222.0.52 () {60 vars in 1482 bytes} [Wed Jul 13 22:12:04 2022] PUT /identity/v3/projects/ed0c80a1d63b412e9936acf0dca3b2ec/users/18b51e7024fc46a8af3644af27ab886f/roles/2a5e86976394498abb6310697cdb3f8b => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 22:12:04.837190 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188397a9-9dac-41dc-bdb8-6eb37a1f4feb admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:12:04.852639 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 792/1588] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jul 13 22:12:04 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 22:12:04.852639 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceaec259-0416-4ca1-b4f0-457ac4f04efb admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:12:04.861930 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ceaec259-0416-4ca1-b4f0-457ac4f04efb admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:12:04.897761 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-188397a9-9dac-41dc-bdb8-6eb37a1f4feb admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:12:04.902407 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f110ed-8a81-49d3-b1e2-d4b53c2e3422 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:12:04.903340 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f110ed-8a81-49d3-b1e2-d4b53c2e3422 admin admin] REQUEST_METHOD: `PUT` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:12:04.903619 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f110ed-8a81-49d3-b1e2-d4b53c2e3422 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:12:04.903893 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34f110ed-8a81-49d3-b1e2-d4b53c2e3422 admin admin] PATH_INFO: `/v3/projects/3035484baa324b01b7aa94b3322d9e6e/users/99a16a550cdf407880fde2ba580fa0c2/roles/2a5e86976394498abb6310697cdb3f8b` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:12:04.918062 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ee717d-4d3f-43b7-a89b-030e91bc1011 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:12:04.918757 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ee717d-4d3f-43b7-a89b-030e91bc1011 admin admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:12:04.918891 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f110ed-8a81-49d3-b1e2-d4b53c2e3422 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3035484baa324b01b7aa94b3322d9e6e, user_id=99a16a550cdf407880fde2ba580fa0c2, role_id=2a5e86976394498abb6310697cdb3f8b)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:12:04.919037 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ee717d-4d3f-43b7-a89b-030e91bc1011 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:12:04.919312 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ee717d-4d3f-43b7-a89b-030e91bc1011 admin admin] PATH_INFO: `/v3/roles` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:12:04.920038 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ee717d-4d3f-43b7-a89b-030e91bc1011 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:12:04.921459 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-34f110ed-8a81-49d3-b1e2-d4b53c2e3422 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:12:04.921664 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ee717d-4d3f-43b7-a89b-030e91bc1011 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:12:04.924672 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:12:04.932895 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 793/1589] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jul 13 22:12:04 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 22:12:04.934664 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ee717d-4d3f-43b7-a89b-030e91bc1011 admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:12:04.940225 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 797/1590] 10.222.0.52 () {60 vars in 1482 bytes} [Wed Jul 13 22:12:04 2022] PUT /identity/v3/projects/3035484baa324b01b7aa94b3322d9e6e/users/99a16a550cdf407880fde2ba580fa0c2/roles/2a5e86976394498abb6310697cdb3f8b => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 22:12:04.941035 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f110ed-8a81-49d3-b1e2-d4b53c2e3422 admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:12:04.944142 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ee717d-4d3f-43b7-a89b-030e91bc1011 admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:12:05.003824 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01561488-ead3-4778-afa3-787c6bbcb1ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:12:05.004497 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01561488-ead3-4778-afa3-787c6bbcb1ce admin admin] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:12:05.004784 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01561488-ead3-4778-afa3-787c6bbcb1ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:12:05.005120 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01561488-ead3-4778-afa3-787c6bbcb1ce admin admin] PATH_INFO: `/v3/users` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:12:05.006044 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01561488-ead3-4778-afa3-787c6bbcb1ce admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:12:05.008030 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-01561488-ead3-4778-afa3-787c6bbcb1ce admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:12:05.010555 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34f110ed-8a81-49d3-b1e2-d4b53c2e3422 admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:12:05.015159 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38e4acd3-540f-4ea9-aa1d-597187150e68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:12:05.015959 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e4acd3-540f-4ea9-aa1d-597187150e68 admin admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:12:05.016137 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e4acd3-540f-4ea9-aa1d-597187150e68 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:12:05.016411 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38e4acd3-540f-4ea9-aa1d-597187150e68 admin admin] PATH_INFO: `/v3/roles` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:12:05.016891 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:INSERT=8,keystone:SELECT=92 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:12:05.017142 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e4acd3-540f-4ea9-aa1d-597187150e68 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:12:05.018825 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38e4acd3-540f-4ea9-aa1d-597187150e68 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:12:05.030480 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 798/1591] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jul 13 22:12:04 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 22:12:05.031727 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0879d7-ae6f-432a-a4b2-37f26217dd19 None None] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:12:05.032010 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0879d7-ae6f-432a-a4b2-37f26217dd19 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:12:05.032282 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-df0879d7-ae6f-432a-a4b2-37f26217dd19 None None] PATH_INFO: `/` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:12:05.034827 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 799/1592] 10.222.0.52 () {54 vars in 961 bytes} [Wed Jul 13 22:12:05 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 22:12:05.035656 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0879d7-ae6f-432a-a4b2-37f26217dd19 None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:12:05.037321 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 794/1593] 10.222.0.52 () {60 vars in 1143 bytes} [Wed Jul 13 22:12:04 2022] POST /identity/v3/users => generated 294 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 22:12:05.038702 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3bfccb-9f02-4fdd-8c76-29d023814c93 None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:12:05.038992 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3bfccb-9f02-4fdd-8c76-29d023814c93 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:12:05.039273 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d3bfccb-9f02-4fdd-8c76-29d023814c93 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:12:05.050329 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-df0879d7-ae6f-432a-a4b2-37f26217dd19 None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:12:05.056114 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ffd77d4-f0b3-4baa-adac-b26061183c19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:12:05.056778 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffd77d4-f0b3-4baa-adac-b26061183c19 admin admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:12:05.056885 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-5d3bfccb-9f02-4fdd-8c76-29d023814c93 None None] MFA Rules not processed for user `cc99c93da5074cf5a48bc2aa9a92c5e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 22:12:05.057056 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffd77d4-f0b3-4baa-adac-b26061183c19 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:12:05.057356 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ffd77d4-f0b3-4baa-adac-b26061183c19 admin admin] PATH_INFO: `/v3/roles` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:12:05.057994 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-5d3bfccb-9f02-4fdd-8c76-29d023814c93 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:12:05.058095 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ffd77d4-f0b3-4baa-adac-b26061183c19 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:12:05.059714 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ffd77d4-f0b3-4baa-adac-b26061183c19 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:12:05.066072 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 795/1594] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jul 13 22:12:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 22:12:05.067113 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3bfccb-9f02-4fdd-8c76-29d023814c93 None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:12:05.074209 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 800/1595] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jul 13 22:12:05 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 22:12:05.074209 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ffd77d4-f0b3-4baa-adac-b26061183c19 admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:12:05.077287 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d3bfccb-9f02-4fdd-8c76-29d023814c93 None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:12:05.081627 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ffd77d4-f0b3-4baa-adac-b26061183c19 admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:12:05.081966 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2471ff87-0ef0-4fa6-9733-f167e3e4bde9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:12:05.082671 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2471ff87-0ef0-4fa6-9733-f167e3e4bde9 admin admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:12:05.082949 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2471ff87-0ef0-4fa6-9733-f167e3e4bde9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:12:05.083299 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2471ff87-0ef0-4fa6-9733-f167e3e4bde9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:12:05.084035 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2471ff87-0ef0-4fa6-9733-f167e3e4bde9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:12:05.085803 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2471ff87-0ef0-4fa6-9733-f167e3e4bde9 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:12:05.086878 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46646db4-7225-4c67-9c93-8fa5be74fd09 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:12:05.087529 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46646db4-7225-4c67-9c93-8fa5be74fd09 admin admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:12:05.087806 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46646db4-7225-4c67-9c93-8fa5be74fd09 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:12:05.088493 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46646db4-7225-4c67-9c93-8fa5be74fd09 admin admin] PATH_INFO: `/v3/roles` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:12:05.088788 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46646db4-7225-4c67-9c93-8fa5be74fd09 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:12:05.090381 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-46646db4-7225-4c67-9c93-8fa5be74fd09 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:12:05.096735 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 796/1596] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jul 13 22:12:05 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 22:12:05.098046 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed0fc71-cad6-4373-8659-f714ccc14064 None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:12:05.098471 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed0fc71-cad6-4373-8659-f714ccc14064 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:12:05.098855 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aed0fc71-cad6-4373-8659-f714ccc14064 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:12:05.104433 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 801/1597] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jul 13 22:12:05 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 22:12:05.104433 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46646db4-7225-4c67-9c93-8fa5be74fd09 admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:12:05.112533 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46646db4-7225-4c67-9c93-8fa5be74fd09 admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:12:05.117775 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c4c4a1-3a11-4dd3-b095-ea48af110d72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:12:05.118414 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c4c4a1-3a11-4dd3-b095-ea48af110d72 admin admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:12:05.118738 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c4c4a1-3a11-4dd3-b095-ea48af110d72 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:12:05.119016 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65c4c4a1-3a11-4dd3-b095-ea48af110d72 admin admin] PATH_INFO: `/v3/roles` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:12:05.119169 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-aed0fc71-cad6-4373-8659-f714ccc14064 None None] MFA Rules not processed for user `cc99c93da5074cf5a48bc2aa9a92c5e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 22:12:05.119736 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c4c4a1-3a11-4dd3-b095-ea48af110d72 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:12:05.120306 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-aed0fc71-cad6-4373-8659-f714ccc14064 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:12:05.121333 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65c4c4a1-3a11-4dd3-b095-ea48af110d72 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:12:05.127979 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 797/1598] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jul 13 22:12:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 22:12:05.129279 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4600472-19ca-49e2-ae13-868e64459ab8 None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:12:05.129562 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4600472-19ca-49e2-ae13-868e64459ab8 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:12:05.129833 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4600472-19ca-49e2-ae13-868e64459ab8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:12:05.134601 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 802/1599] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jul 13 22:12:05 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 22:12:05.134601 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c4c4a1-3a11-4dd3-b095-ea48af110d72 admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:12:05.142521 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65c4c4a1-3a11-4dd3-b095-ea48af110d72 admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:12:05.147765 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209c3927-35b1-46d0-b4ee-6a5a0d8b95d7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:12:05.148423 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209c3927-35b1-46d0-b4ee-6a5a0d8b95d7 admin admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:12:05.148708 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209c3927-35b1-46d0-b4ee-6a5a0d8b95d7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:12:05.149051 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-209c3927-35b1-46d0-b4ee-6a5a0d8b95d7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:12:05.149707 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209c3927-35b1-46d0-b4ee-6a5a0d8b95d7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:12:05.151373 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-209c3927-35b1-46d0-b4ee-6a5a0d8b95d7 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:12:05.164513 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 803/1600] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jul 13 22:12:05 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 22:12:05.164513 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209c3927-35b1-46d0-b4ee-6a5a0d8b95d7 admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:12:05.170050 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-b4600472-19ca-49e2-ae13-868e64459ab8 None None] MFA Rules not processed for user `f84281091687464990ffbca590b9343b`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 22:12:05.171286 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-b4600472-19ca-49e2-ae13-868e64459ab8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:12:05.172191 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-209c3927-35b1-46d0-b4ee-6a5a0d8b95d7 admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:12:05.176829 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba30353d-279f-4bbc-a765-e4d59f90e3cc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:12:05.178264 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba30353d-279f-4bbc-a765-e4d59f90e3cc admin admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:12:05.178576 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba30353d-279f-4bbc-a765-e4d59f90e3cc admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:12:05.178852 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba30353d-279f-4bbc-a765-e4d59f90e3cc admin admin] PATH_INFO: `/v3/roles` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:12:05.179575 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba30353d-279f-4bbc-a765-e4d59f90e3cc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:12:05.181151 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba30353d-279f-4bbc-a765-e4d59f90e3cc admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:12:05.191602 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 804/1601] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jul 13 22:12:05 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 22:12:05.192633 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba30353d-279f-4bbc-a765-e4d59f90e3cc admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:12:05.244436 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 798/1602] 10.222.0.52 () {58 vars in 1058 bytes} [Wed Jul 13 22:12:05 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 22:12:05.247428 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4600472-19ca-49e2-ae13-868e64459ab8 None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:12:05.248443 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba30353d-279f-4bbc-a765-e4d59f90e3cc admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:12:05.252203 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2496558-634f-443d-b8a1-406c65a5c9d7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '847ec9a1bd4d4dec804318a5c845895a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417e3e2f1333481795a089bef2e24bea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:12:05.253145 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2496558-634f-443d-b8a1-406c65a5c9d7 service neutron] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:12:05.253541 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2496558-634f-443d-b8a1-406c65a5c9d7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:12:05.254401 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f2496558-634f-443d-b8a1-406c65a5c9d7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:12:05.258401 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4600472-19ca-49e2-ae13-868e64459ab8 None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:12:05.263173 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2534c88d-3bc5-4527-9d97-76dab934e8bc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:12:05.264069 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2534c88d-3bc5-4527-9d97-76dab934e8bc admin admin] REQUEST_METHOD: `PUT` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:12:05.264352 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2534c88d-3bc5-4527-9d97-76dab934e8bc admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:12:05.264543 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2496558-634f-443d-b8a1-406c65a5c9d7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:12:05.264661 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2534c88d-3bc5-4527-9d97-76dab934e8bc admin admin] PATH_INFO: `/v3/projects/75f4f5476c7e451a82f1cfc82a0ee689/users/45b5d8b0ba324f629f697cf0de5918bb/roles/2a5e86976394498abb6310697cdb3f8b` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:12:05.265979 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f2496558-634f-443d-b8a1-406c65a5c9d7 service neutron] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:12:05.275981 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 805/1603] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jul 13 22:12:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 22:12:05.276711 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2496558-634f-443d-b8a1-406c65a5c9d7 service neutron] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:12:05.282752 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2534c88d-3bc5-4527-9d97-76dab934e8bc admin admin] RBAC: Authorizing `identity:create_grant(project_id=75f4f5476c7e451a82f1cfc82a0ee689, user_id=45b5d8b0ba324f629f697cf0de5918bb, role_id=2a5e86976394498abb6310697cdb3f8b)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:12:05.285529 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2534c88d-3bc5-4527-9d97-76dab934e8bc admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:12:05.287536 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f2496558-634f-443d-b8a1-406c65a5c9d7 service neutron] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:12:05.293147 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3815abb-5ced-43d8-acf2-a51ab30244da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:12:05.294136 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3815abb-5ced-43d8-acf2-a51ab30244da admin admin] REQUEST_METHOD: `PUT` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:12:05.294418 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3815abb-5ced-43d8-acf2-a51ab30244da admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:12:05.294721 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3815abb-5ced-43d8-acf2-a51ab30244da admin admin] PATH_INFO: `/v3/projects/fd35df3b5fdb45499e9ee5d6024a6054/users/064dcac8f59c431cb7d6848f4aaac347/roles/2a5e86976394498abb6310697cdb3f8b` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:12:05.304136 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 799/1604] 10.222.0.52 () {60 vars in 1482 bytes} [Wed Jul 13 22:12:05 2022] PUT /identity/v3/projects/75f4f5476c7e451a82f1cfc82a0ee689/users/45b5d8b0ba324f629f697cf0de5918bb/roles/2a5e86976394498abb6310697cdb3f8b => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 22:12:05.304965 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2534c88d-3bc5-4527-9d97-76dab934e8bc admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:12:05.315110 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3815abb-5ced-43d8-acf2-a51ab30244da admin admin] RBAC: Authorizing `identity:create_grant(project_id=fd35df3b5fdb45499e9ee5d6024a6054, user_id=064dcac8f59c431cb7d6848f4aaac347, role_id=2a5e86976394498abb6310697cdb3f8b)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:12:05.318020 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3815abb-5ced-43d8-acf2-a51ab30244da admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:12:05.337277 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 806/1605] 10.222.0.52 () {60 vars in 1482 bytes} [Wed Jul 13 22:12:05 2022] PUT /identity/v3/projects/fd35df3b5fdb45499e9ee5d6024a6054/users/064dcac8f59c431cb7d6848f4aaac347/roles/2a5e86976394498abb6310697cdb3f8b => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 22:12:05.338200 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3815abb-5ced-43d8-acf2-a51ab30244da admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:12:05.364313 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:12:05.383563 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2534c88d-3bc5-4527-9d97-76dab934e8bc admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:12:05.418517 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:INSERT=2,keystone:SELECT=98 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:12:05.436633 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3815abb-5ced-43d8-acf2-a51ab30244da admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:12:05.441157 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d6d200-3367-44d8-bf7a-c31e0f4da92a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '847ec9a1bd4d4dec804318a5c845895a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417e3e2f1333481795a089bef2e24bea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:12:05.441862 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d6d200-3367-44d8-bf7a-c31e0f4da92a service neutron] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:12:05.442108 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d6d200-3367-44d8-bf7a-c31e0f4da92a service neutron] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:12:05.442402 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e4d6d200-3367-44d8-bf7a-c31e0f4da92a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:12:05.464446 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5f6b60-b4ac-4aa2-9cab-6681621ebf34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:12:05.465596 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5f6b60-b4ac-4aa2-9cab-6681621ebf34 admin admin] REQUEST_METHOD: `PUT` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:12:05.466178 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5f6b60-b4ac-4aa2-9cab-6681621ebf34 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:12:05.466756 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d5f6b60-b4ac-4aa2-9cab-6681621ebf34 admin admin] PATH_INFO: `/v3/projects/46d57b6c4d1340649e0de1a8e8ec09f4/users/7a993519e77740f5bd08fd11906ba50b/roles/2a5e86976394498abb6310697cdb3f8b` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:12:05.485313 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5f6b60-b4ac-4aa2-9cab-6681621ebf34 admin admin] RBAC: Authorizing `identity:create_grant(project_id=46d57b6c4d1340649e0de1a8e8ec09f4, user_id=7a993519e77740f5bd08fd11906ba50b, role_id=2a5e86976394498abb6310697cdb3f8b)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:12:05.488521 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9d5f6b60-b4ac-4aa2-9cab-6681621ebf34 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:12:05.507014 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 800/1606] 10.222.0.52 () {60 vars in 1482 bytes} [Wed Jul 13 22:12:05 2022] PUT /identity/v3/projects/46d57b6c4d1340649e0de1a8e8ec09f4/users/7a993519e77740f5bd08fd11906ba50b/roles/2a5e86976394498abb6310697cdb3f8b => generated 0 bytes in 202 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 22:12:05.519364 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5f6b60-b4ac-4aa2-9cab-6681621ebf34 admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:12:05.564504 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d6d200-3367-44d8-bf7a-c31e0f4da92a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:12:05.567344 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e4d6d200-3367-44d8-bf7a-c31e0f4da92a service neutron] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:12:05.587265 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 807/1607] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jul 13 22:12:05 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 249 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 22:12:05.588660 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d6d200-3367-44d8-bf7a-c31e0f4da92a service neutron] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:12:05.599209 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d5f6b60-b4ac-4aa2-9cab-6681621ebf34 admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:12:05.604436 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93a6a54-49d1-4b94-bc87-d10168f56d7f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:12:05.605120 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93a6a54-49d1-4b94-bc87-d10168f56d7f admin admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:12:05.605405 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93a6a54-49d1-4b94-bc87-d10168f56d7f admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:12:05.605570 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e4d6d200-3367-44d8-bf7a-c31e0f4da92a service neutron] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:12:05.605681 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c93a6a54-49d1-4b94-bc87-d10168f56d7f admin admin] PATH_INFO: `/v3/roles` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:12:05.606409 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c93a6a54-49d1-4b94-bc87-d10168f56d7f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:12:05.608178 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c93a6a54-49d1-4b94-bc87-d10168f56d7f admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:12:05.613527 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-271dd52e-d1f7-4a3f-ba2c-40e551b4ad0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:12:05.615075 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271dd52e-d1f7-4a3f-ba2c-40e551b4ad0e admin admin] REQUEST_METHOD: `PUT` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:12:05.615556 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271dd52e-d1f7-4a3f-ba2c-40e551b4ad0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:12:05.616389 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-271dd52e-d1f7-4a3f-ba2c-40e551b4ad0e admin admin] PATH_INFO: `/v3/projects/3ae798b1b7bf4730bc5186e196db3d6e/users/326dab89fb8c4846b4bee1bc63178260/roles/2a5e86976394498abb6310697cdb3f8b` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:12:05.622890 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 801/1608] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jul 13 22:12:05 2022] GET /identity/v3/roles => generated 1420 bytes in 112 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 22:12:05.622890 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93a6a54-49d1-4b94-bc87-d10168f56d7f admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:12:05.640888 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-271dd52e-d1f7-4a3f-ba2c-40e551b4ad0e admin admin] RBAC: Authorizing `identity:create_grant(project_id=3ae798b1b7bf4730bc5186e196db3d6e, user_id=326dab89fb8c4846b4bee1bc63178260, role_id=2a5e86976394498abb6310697cdb3f8b)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:12:05.644136 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-271dd52e-d1f7-4a3f-ba2c-40e551b4ad0e admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:12:05.673511 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 808/1609] 10.222.0.52 () {60 vars in 1482 bytes} [Wed Jul 13 22:12:05 2022] PUT /identity/v3/projects/3ae798b1b7bf4730bc5186e196db3d6e/users/326dab89fb8c4846b4bee1bc63178260/roles/2a5e86976394498abb6310697cdb3f8b => generated 0 bytes in 85 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 22:12:05.674570 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0e736e-d252-4671-bb2c-bfb645ad354a None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:12:05.674789 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0e736e-d252-4671-bb2c-bfb645ad354a None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:12:05.675223 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d0e736e-d252-4671-bb2c-bfb645ad354a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:12:05.696559 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-8d0e736e-d252-4671-bb2c-bfb645ad354a None None] MFA Rules not processed for user `cc99c93da5074cf5a48bc2aa9a92c5e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 22:12:05.698173 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-8d0e736e-d252-4671-bb2c-bfb645ad354a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:12:05.736346 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c93a6a54-49d1-4b94-bc87-d10168f56d7f admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:12:05.779059 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 809/1610] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jul 13 22:12:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 22:12:05.780828 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95d3011-d9aa-4d8a-a34d-4b9575fbca2a None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:12:05.781293 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95d3011-d9aa-4d8a-a34d-4b9575fbca2a None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:12:05.782236 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e95d3011-d9aa-4d8a-a34d-4b9575fbca2a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:12:05.797670 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:12:05.810310 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-e95d3011-d9aa-4d8a-a34d-4b9575fbca2a None None] MFA Rules not processed for user `cc99c93da5074cf5a48bc2aa9a92c5e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 22:12:05.811813 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-e95d3011-d9aa-4d8a-a34d-4b9575fbca2a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:12:05.825082 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 810/1611] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jul 13 22:12:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 44 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 22:12:05.826940 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bda8ea-31fe-46d2-b994-e00369774d8d None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:12:05.827232 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bda8ea-31fe-46d2-b994-e00369774d8d None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:12:05.827596 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-37bda8ea-31fe-46d2-b994-e00369774d8d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:12:05.830716 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c1cf18b-c962-433f-bf91-71c56fca21e9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84281091687464990ffbca590b9343b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417e3e2f1333481795a089bef2e24bea', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:12:05.831729 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1cf18b-c962-433f-bf91-71c56fca21e9 service nova] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:12:05.832099 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1cf18b-c962-433f-bf91-71c56fca21e9 service nova] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:12:05.832530 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c1cf18b-c962-433f-bf91-71c56fca21e9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:12:05.867801 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-37bda8ea-31fe-46d2-b994-e00369774d8d None None] MFA Rules not processed for user `7a993519e77740f5bd08fd11906ba50b`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 22:12:05.869398 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-37bda8ea-31fe-46d2-b994-e00369774d8d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:12:05.900615 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:12:05.966114 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1cf18b-c962-433f-bf91-71c56fca21e9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:12:05.968943 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c1cf18b-c962-433f-bf91-71c56fca21e9 service nova] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:12:05.973396 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 811/1612] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jul 13 22:12:05 2022] POST /identity/v3/auth/tokens => generated 3234 bytes in 148 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 22:12:05.974998 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37bda8ea-31fe-46d2-b994-e00369774d8d None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:12:05.987067 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 802/1613] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jul 13 22:12:05 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 367 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 22:12:05.989671 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1956ecfa-dd6a-46a6-8395-e7691a5cccd2 None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:12:05.990135 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1956ecfa-dd6a-46a6-8395-e7691a5cccd2 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:12:05.990595 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1956ecfa-dd6a-46a6-8395-e7691a5cccd2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:12:06.004094 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-37bda8ea-31fe-46d2-b994-e00369774d8d None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:12:06.022200 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac965d4e-a850-4ac4-83c8-cd491a178aeb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:12:06.023246 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac965d4e-a850-4ac4-83c8-cd491a178aeb admin admin] REQUEST_METHOD: `PUT` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:12:06.023988 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac965d4e-a850-4ac4-83c8-cd491a178aeb admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:12:06.024330 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ac965d4e-a850-4ac4-83c8-cd491a178aeb admin admin] PATH_INFO: `/v3/projects/7bd88efc54f94e509196de33857ee865/users/83d2f82cf096424aa4979f443a2729d6/roles/2a5e86976394498abb6310697cdb3f8b` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:12:06.028874 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-1956ecfa-dd6a-46a6-8395-e7691a5cccd2 None None] MFA Rules not processed for user `326dab89fb8c4846b4bee1bc63178260`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 22:12:06.030951 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-1956ecfa-dd6a-46a6-8395-e7691a5cccd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:12:06.051650 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac965d4e-a850-4ac4-83c8-cd491a178aeb admin admin] RBAC: Authorizing `identity:create_grant(project_id=7bd88efc54f94e509196de33857ee865, user_id=83d2f82cf096424aa4979f443a2729d6, role_id=2a5e86976394498abb6310697cdb3f8b)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:12:06.055257 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ac965d4e-a850-4ac4-83c8-cd491a178aeb admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:12:06.076708 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 812/1614] 10.222.0.52 () {60 vars in 1482 bytes} [Wed Jul 13 22:12:05 2022] PUT /identity/v3/projects/7bd88efc54f94e509196de33857ee865/users/83d2f82cf096424aa4979f443a2729d6/roles/2a5e86976394498abb6310697cdb3f8b => generated 0 bytes in 102 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 22:12:06.077825 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac965d4e-a850-4ac4-83c8-cd491a178aeb admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:12:06.088228 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-ac965d4e-a850-4ac4-83c8-cd491a178aeb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:12:06.168189 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 803/1615] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jul 13 22:12:05 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 22:12:06.168836 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1956ecfa-dd6a-46a6-8395-e7691a5cccd2 None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:12:06.175208 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-1956ecfa-dd6a-46a6-8395-e7691a5cccd2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:12:06.256613 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ac965d4e-a850-4ac4-83c8-cd491a178aeb admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:12:06.262508 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4d7890-876c-42ab-93d3-7d98cae5e617 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '847ec9a1bd4d4dec804318a5c845895a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417e3e2f1333481795a089bef2e24bea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:12:06.263392 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4d7890-876c-42ab-93d3-7d98cae5e617 service neutron] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:12:06.263920 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4d7890-876c-42ab-93d3-7d98cae5e617 service neutron] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:12:06.264327 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d4d7890-876c-42ab-93d3-7d98cae5e617 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:12:06.294222 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1956ecfa-dd6a-46a6-8395-e7691a5cccd2 None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:12:06.300961 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6709c50a-b011-46e1-9d27-a5ed799b7042 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '847ec9a1bd4d4dec804318a5c845895a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417e3e2f1333481795a089bef2e24bea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:12:06.302454 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6709c50a-b011-46e1-9d27-a5ed799b7042 service neutron] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:12:06.302937 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6709c50a-b011-46e1-9d27-a5ed799b7042 service neutron] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:12:06.303407 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6709c50a-b011-46e1-9d27-a5ed799b7042 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:12:06.344496 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4d7890-876c-42ab-93d3-7d98cae5e617 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:12:06.346915 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7d4d7890-876c-42ab-93d3-7d98cae5e617 service neutron] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:12:06.358999 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 813/1616] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jul 13 22:12:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 282 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 22:12:06.360784 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4d7890-876c-42ab-93d3-7d98cae5e617 service neutron] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:12:06.372652 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:12:06.461293 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6709c50a-b011-46e1-9d27-a5ed799b7042 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:12:06.463839 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6709c50a-b011-46e1-9d27-a5ed799b7042 service neutron] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:12:06.478888 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 804/1617] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jul 13 22:12:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 310 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 22:12:06.480589 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d638b4b-c0a1-45f3-8c88-45e7f2f8334a None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:12:06.481127 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d638b4b-c0a1-45f3-8c88-45e7f2f8334a None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:12:06.481689 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d638b4b-c0a1-45f3-8c88-45e7f2f8334a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:12:06.492719 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d4d7890-876c-42ab-93d3-7d98cae5e617 service neutron] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:12:06.498215 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d474493c-6075-47f0-aed4-6b09febc332c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc77e1ea1ada4700b809beb1c980c5e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417e3e2f1333481795a089bef2e24bea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:12:06.499200 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d474493c-6075-47f0-aed4-6b09febc332c service cinder] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:12:06.499672 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d474493c-6075-47f0-aed4-6b09febc332c service cinder] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:12:06.500143 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d474493c-6075-47f0-aed4-6b09febc332c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:12:06.508305 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:12:06.521469 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-1d638b4b-c0a1-45f3-8c88-45e7f2f8334a None None] MFA Rules not processed for user `cc99c93da5074cf5a48bc2aa9a92c5e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 22:12:06.527520 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-1d638b4b-c0a1-45f3-8c88-45e7f2f8334a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:12:06.553261 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 805/1618] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jul 13 22:12:06 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 74 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 22:12:06.554757 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5801e15-015e-47a4-b935-2b15ec833e1c None None] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:12:06.556839 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5801e15-015e-47a4-b935-2b15ec833e1c None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:12:06.557090 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5801e15-015e-47a4-b935-2b15ec833e1c None None] PATH_INFO: `/` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:12:06.560443 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 806/1619] 10.222.0.52 () {54 vars in 961 bytes} [Wed Jul 13 22:12:06 2022] GET /identity => generated 270 bytes in 7 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 22:12:06.563894 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c0bd0c-5058-428c-9239-51eeb49ee635 None None] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:12:06.565698 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c0bd0c-5058-428c-9239-51eeb49ee635 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:12:06.565923 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88c0bd0c-5058-428c-9239-51eeb49ee635 None None] PATH_INFO: `/` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:12:06.580052 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 807/1620] 10.222.0.52 () {54 vars in 912 bytes} [Wed Jul 13 22:12:06 2022] GET /identity => generated 270 bytes in 19 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 22:12:06.581573 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6062fcf-646b-4842-b41a-7df80b4eb0d1 None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:12:06.581959 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6062fcf-646b-4842-b41a-7df80b4eb0d1 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:12:06.582690 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6062fcf-646b-4842-b41a-7df80b4eb0d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:12:06.625937 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-c6062fcf-646b-4842-b41a-7df80b4eb0d1 None None] MFA Rules not processed for user `f84281091687464990ffbca590b9343b`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 22:12:06.627189 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-c6062fcf-646b-4842-b41a-7df80b4eb0d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:12:06.629061 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d474493c-6075-47f0-aed4-6b09febc332c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:12:06.630804 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d474493c-6075-47f0-aed4-6b09febc332c service cinder] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:12:06.648757 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 814/1621] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jul 13 22:12:06 2022] GET /identity/v3/auth/tokens => generated 3234 bytes in 289 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 22:12:06.650140 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf4b7a2-cb4b-4e27-9363-7d8851b46e45 None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:12:06.650417 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf4b7a2-cb4b-4e27-9363-7d8851b46e45 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:12:06.650713 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcf4b7a2-cb4b-4e27-9363-7d8851b46e45 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:12:06.714431 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-dcf4b7a2-cb4b-4e27-9363-7d8851b46e45 None None] MFA Rules not processed for user `f84281091687464990ffbca590b9343b`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 22:12:06.719727 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-dcf4b7a2-cb4b-4e27-9363-7d8851b46e45 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:12:06.788476 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 808/1622] 10.222.0.52 () {58 vars in 1058 bytes} [Wed Jul 13 22:12:06 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 22:12:06.789210 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6062fcf-646b-4842-b41a-7df80b4eb0d1 None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:12:06.810384 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6062fcf-646b-4842-b41a-7df80b4eb0d1 None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:12:06.819949 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af28c76e-81c4-4896-abf4-87ba72c0f509 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '847ec9a1bd4d4dec804318a5c845895a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417e3e2f1333481795a089bef2e24bea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:12:06.820629 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af28c76e-81c4-4896-abf4-87ba72c0f509 service neutron] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:12:06.822076 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af28c76e-81c4-4896-abf4-87ba72c0f509 service neutron] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:12:06.823222 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af28c76e-81c4-4896-abf4-87ba72c0f509 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:12:06.844794 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af28c76e-81c4-4896-abf4-87ba72c0f509 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:12:06.848559 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af28c76e-81c4-4896-abf4-87ba72c0f509 service neutron] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:12:06.879152 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 809/1623] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jul 13 22:12:06 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 22:12:06.879912 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af28c76e-81c4-4896-abf4-87ba72c0f509 service neutron] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:12:06.907872 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af28c76e-81c4-4896-abf4-87ba72c0f509 service neutron] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:12:06.919762 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b73f7526-467b-4e93-baa1-ee58ed28664d service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84281091687464990ffbca590b9343b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417e3e2f1333481795a089bef2e24bea', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:12:06.922589 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73f7526-467b-4e93-baa1-ee58ed28664d service nova] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:12:06.922898 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73f7526-467b-4e93-baa1-ee58ed28664d service nova] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:12:06.923182 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b73f7526-467b-4e93-baa1-ee58ed28664d service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:12:06.962772 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 815/1624] 10.222.0.52 () {58 vars in 1009 bytes} [Wed Jul 13 22:12:06 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 313 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 22:12:06.971959 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf4b7a2-cb4b-4e27-9363-7d8851b46e45 None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:12:06.988374 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcf4b7a2-cb4b-4e27-9363-7d8851b46e45 None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:12:07.001238 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-75eded83-05b5-4204-a348-655a253f7131 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc77e1ea1ada4700b809beb1c980c5e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417e3e2f1333481795a089bef2e24bea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:12:07.003013 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75eded83-05b5-4204-a348-655a253f7131 service cinder] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:12:07.004329 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75eded83-05b5-4204-a348-655a253f7131 service cinder] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:12:07.005811 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-75eded83-05b5-4204-a348-655a253f7131 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:12:07.043937 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75eded83-05b5-4204-a348-655a253f7131 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:12:07.045726 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-75eded83-05b5-4204-a348-655a253f7131 service cinder] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:12:07.050516 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73f7526-467b-4e93-baa1-ee58ed28664d service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:12:07.053969 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b73f7526-467b-4e93-baa1-ee58ed28664d service nova] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:12:07.065191 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 816/1625] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jul 13 22:12:06 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 22:12:07.081622 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 810/1626] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jul 13 22:12:06 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 198 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 22:12:07.627240 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c7d7eb-12af-4d77-b085-c1bf4a4a3e54 None None] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:12:07.628379 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c7d7eb-12af-4d77-b085-c1bf4a4a3e54 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:12:07.628800 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1c7d7eb-12af-4d77-b085-c1bf4a4a3e54 None None] PATH_INFO: `/` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:12:07.630613 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 817/1627] 10.222.0.52 () {54 vars in 912 bytes} [Wed Jul 13 22:12:07 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 22:12:07.638088 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bdcbaa-1d69-4e11-a1d8-bfbf1b5efb4b None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:12:07.638641 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bdcbaa-1d69-4e11-a1d8-bfbf1b5efb4b None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:12:07.638725 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7bdcbaa-1d69-4e11-a1d8-bfbf1b5efb4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:12:07.670817 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-a7bdcbaa-1d69-4e11-a1d8-bfbf1b5efb4b None None] MFA Rules not processed for user `f84281091687464990ffbca590b9343b`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 22:12:07.673201 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-a7bdcbaa-1d69-4e11-a1d8-bfbf1b5efb4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:12:07.702519 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 811/1628] 10.222.0.52 () {58 vars in 1009 bytes} [Wed Jul 13 22:12:07 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 65 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 22:12:07.717244 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c7d7eb-12af-4d77-b085-c1bf4a4a3e54 None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:12:07.736335 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1c7d7eb-12af-4d77-b085-c1bf4a4a3e54 None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:12:07.743665 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14a61fd7-0524-48dc-bfc2-3bd2c1a73075 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc77e1ea1ada4700b809beb1c980c5e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417e3e2f1333481795a089bef2e24bea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:12:07.744502 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a61fd7-0524-48dc-bfc2-3bd2c1a73075 service cinder] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:12:07.744852 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a61fd7-0524-48dc-bfc2-3bd2c1a73075 service cinder] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:12:07.745197 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14a61fd7-0524-48dc-bfc2-3bd2c1a73075 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:12:07.758900 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a61fd7-0524-48dc-bfc2-3bd2c1a73075 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:12:07.760777 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-14a61fd7-0524-48dc-bfc2-3bd2c1a73075 service cinder] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:12:07.781189 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 818/1629] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jul 13 22:12:07 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 64 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 22:12:08.038622 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7bdcbaa-1d69-4e11-a1d8-bfbf1b5efb4b None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:12:08.069621 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7bdcbaa-1d69-4e11-a1d8-bfbf1b5efb4b None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:12:08.101449 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb8f12e-a635-4e48-9435-bd3fef9c9f0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:12:08.106107 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb8f12e-a635-4e48-9435-bd3fef9c9f0f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:12:08.108322 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb8f12e-a635-4e48-9435-bd3fef9c9f0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:12:08.109609 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bfb8f12e-a635-4e48-9435-bd3fef9c9f0f admin admin] PATH_INFO: `/v3/users/326dab89fb8c4846b4bee1bc63178260` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:12:08.118953 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb8f12e-a635-4e48-9435-bd3fef9c9f0f admin admin] RBAC: Authorizing `identity:delete_user(user_id=326dab89fb8c4846b4bee1bc63178260)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:12:08.119970 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: WARNING py.warnings [None req-bfb8f12e-a635-4e48-9435-bd3fef9c9f0f admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_user": "(role:admin and system_scope:all) or (role:admin and token.domain.id:%(target.user.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 13 22:12:08.119970 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: warnings.warn(msg) Jul 13 22:12:08.119970 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: Jul 13 22:12:08.122178 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bfb8f12e-a635-4e48-9435-bd3fef9c9f0f admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:12:08.207924 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:12:08.269639 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-bfb8f12e-a635-4e48-9435-bd3fef9c9f0f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '326dab89fb8c4846b4bee1bc63178260'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:12:08.292205 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-bfb8f12e-a635-4e48-9435-bd3fef9c9f0f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '326dab89fb8c4846b4bee1bc63178260'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:12:08.321735 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-bfb8f12e-a635-4e48-9435-bd3fef9c9f0f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '326dab89fb8c4846b4bee1bc63178260'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:12:08.323243 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-bfb8f12e-a635-4e48-9435-bd3fef9c9f0f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '326dab89fb8c4846b4bee1bc63178260'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:12:08.323946 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-bfb8f12e-a635-4e48-9435-bd3fef9c9f0f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '326dab89fb8c4846b4bee1bc63178260'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:12:08.348532 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 812/1630] 10.222.0.52 () {58 vars in 1223 bytes} [Wed Jul 13 22:12:08 2022] DELETE /identity/v3/users/326dab89fb8c4846b4bee1bc63178260 => generated 0 bytes in 311 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 22:12:08.354091 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c665cc-d268-4540-a0c9-28ebcb260f5c None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:12:08.354382 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c665cc-d268-4540-a0c9-28ebcb260f5c None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:12:08.354686 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c665cc-d268-4540-a0c9-28ebcb260f5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:12:08.380917 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-08c665cc-d268-4540-a0c9-28ebcb260f5c None None] MFA Rules not processed for user `cc99c93da5074cf5a48bc2aa9a92c5e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 22:12:08.382351 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-08c665cc-d268-4540-a0c9-28ebcb260f5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:12:08.519328 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 819/1631] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jul 13 22:12:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 22:12:08.588571 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb8f12e-a635-4e48-9435-bd3fef9c9f0f admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:12:08.590415 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-bfb8f12e-a635-4e48-9435-bd3fef9c9f0f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:12:08.680993 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bfb8f12e-a635-4e48-9435-bd3fef9c9f0f admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:12:08.687092 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7a92973-5efa-4e79-9ba9-344acf4ab06e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '847ec9a1bd4d4dec804318a5c845895a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417e3e2f1333481795a089bef2e24bea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:12:08.688359 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a92973-5efa-4e79-9ba9-344acf4ab06e service neutron] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:12:08.688859 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a92973-5efa-4e79-9ba9-344acf4ab06e service neutron] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:12:08.689370 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7a92973-5efa-4e79-9ba9-344acf4ab06e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:12:08.702765 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a92973-5efa-4e79-9ba9-344acf4ab06e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:12:08.706297 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7a92973-5efa-4e79-9ba9-344acf4ab06e service neutron] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:12:08.722595 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 813/1632] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jul 13 22:12:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 134 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 22:12:11.385040 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c665cc-d268-4540-a0c9-28ebcb260f5c None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:12:11.386965 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-08c665cc-d268-4540-a0c9-28ebcb260f5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:12:11.407562 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c665cc-d268-4540-a0c9-28ebcb260f5c None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:12:11.419727 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18eaa0ae-c0af-4f38-8268-83c631f918b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:12:11.422307 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18eaa0ae-c0af-4f38-8268-83c631f918b7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:12:11.424027 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18eaa0ae-c0af-4f38-8268-83c631f918b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:12:11.424451 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18eaa0ae-c0af-4f38-8268-83c631f918b7 admin admin] PATH_INFO: `/v3/projects/3ae798b1b7bf4730bc5186e196db3d6e` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:12:11.427473 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18eaa0ae-c0af-4f38-8268-83c631f918b7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3ae798b1b7bf4730bc5186e196db3d6e)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:12:11.431111 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: WARNING py.warnings [None req-18eaa0ae-c0af-4f38-8268-83c631f918b7 admin admin] /usr/local/lib/python3.8/dist-packages/oslo_policy/policy.py:1129: UserWarning: Policy "identity:delete_project": "(role:admin and system_scope:all) or (role:admin and domain_id:%(target.project.domain_id)s)" failed scope check. The token used to make the request was project scoped but the policy requires ['system', 'domain'] scope. This behavior may change in the future where using the intended scope is required Jul 13 22:12:11.431111 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: warnings.warn(msg) Jul 13 22:12:11.431111 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: Jul 13 22:12:11.435266 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18eaa0ae-c0af-4f38-8268-83c631f918b7 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:12:11.459640 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:12:11.601613 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-18eaa0ae-c0af-4f38-8268-83c631f918b7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3ae798b1b7bf4730bc5186e196db3d6e', 'request_id': 'req-18eaa0ae-c0af-4f38-8268-83c631f918b7'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:12:11.632348 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-18eaa0ae-c0af-4f38-8268-83c631f918b7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3ae798b1b7bf4730bc5186e196db3d6e', 'request_id': 'req-18eaa0ae-c0af-4f38-8268-83c631f918b7'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:12:11.647921 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-18eaa0ae-c0af-4f38-8268-83c631f918b7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3ae798b1b7bf4730bc5186e196db3d6e', 'request_id': 'req-18eaa0ae-c0af-4f38-8268-83c631f918b7'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:12:11.673077 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-18eaa0ae-c0af-4f38-8268-83c631f918b7 admin admin] The token cache is being invalidate because project 3ae798b1b7bf4730bc5186e196db3d6e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80381) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 22:12:11.673373 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-18eaa0ae-c0af-4f38-8268-83c631f918b7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:12:11.676140 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-18eaa0ae-c0af-4f38-8268-83c631f918b7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:12:11.679636 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 820/1633] 10.222.0.52 () {58 vars in 1232 bytes} [Wed Jul 13 22:12:11 2022] DELETE /identity/v3/projects/3ae798b1b7bf4730bc5186e196db3d6e => generated 0 bytes in 295 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 22:12:11.685512 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a642d3b6-9255-4090-ac8b-faf38d542b11 None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:12:11.686015 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a642d3b6-9255-4090-ac8b-faf38d542b11 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:12:11.686181 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a642d3b6-9255-4090-ac8b-faf38d542b11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:12:11.719673 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-a642d3b6-9255-4090-ac8b-faf38d542b11 None None] MFA Rules not processed for user `cc99c93da5074cf5a48bc2aa9a92c5e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 22:12:11.721202 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-a642d3b6-9255-4090-ac8b-faf38d542b11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:12:11.849969 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 814/1634] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jul 13 22:12:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 160 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 22:12:11.851818 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6875c8b-034d-41a0-8988-53477c069e27 None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:12:11.852174 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6875c8b-034d-41a0-8988-53477c069e27 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:12:11.852520 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6875c8b-034d-41a0-8988-53477c069e27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:12:11.883284 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-f6875c8b-034d-41a0-8988-53477c069e27 None None] MFA Rules not processed for user `cc99c93da5074cf5a48bc2aa9a92c5e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 22:12:11.885388 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-f6875c8b-034d-41a0-8988-53477c069e27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:12:11.903152 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 821/1635] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jul 13 22:12:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 52 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 22:12:11.907429 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a642d3b6-9255-4090-ac8b-faf38d542b11 None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:12:11.922944 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a642d3b6-9255-4090-ac8b-faf38d542b11 None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:12:11.932244 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be1a738-9147-4faf-8582-a78ec21f3dac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:12:11.933982 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be1a738-9147-4faf-8582-a78ec21f3dac admin admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:12:11.934631 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be1a738-9147-4faf-8582-a78ec21f3dac admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:12:11.935246 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3be1a738-9147-4faf-8582-a78ec21f3dac admin admin] PATH_INFO: `/v3/domains` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:12:11.939110 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be1a738-9147-4faf-8582-a78ec21f3dac admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 22:12:11.941516 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be1a738-9147-4faf-8582-a78ec21f3dac admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:12:11.943919 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3be1a738-9147-4faf-8582-a78ec21f3dac admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:12:11.964018 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 815/1636] 10.222.0.52 () {58 vars in 1152 bytes} [Wed Jul 13 22:12:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 22:12:11.968270 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6875c8b-034d-41a0-8988-53477c069e27 None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:12:11.987103 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6875c8b-034d-41a0-8988-53477c069e27 None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:12:12.003555 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be52182-d7f4-42d6-90da-6f3d16dfc634 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:12:12.005862 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be52182-d7f4-42d6-90da-6f3d16dfc634 admin admin] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:12:12.006367 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be52182-d7f4-42d6-90da-6f3d16dfc634 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:12:12.006823 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7be52182-d7f4-42d6-90da-6f3d16dfc634 admin admin] PATH_INFO: `/v3/projects` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:12:12.011056 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be52182-d7f4-42d6-90da-6f3d16dfc634 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:12:12.016817 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7be52182-d7f4-42d6-90da-6f3d16dfc634 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:12:12.111725 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be1a738-9147-4faf-8582-a78ec21f3dac admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:12:12.112584 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 822/1637] 10.222.0.52 () {60 vars in 1152 bytes} [Wed Jul 13 22:12:11 2022] POST /identity/v3/projects => generated 365 bytes in 143 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 22:12:12.125447 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3be1a738-9147-4faf-8582-a78ec21f3dac admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:12:12.256868 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=93,keystone:DELETE=6,keystone:INSERT=1 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:12:12.336246 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5a57e3-d2ea-4876-aa0e-fecb35515c19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:12:12.337136 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5a57e3-d2ea-4876-aa0e-fecb35515c19 admin admin] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:12:12.338691 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5a57e3-d2ea-4876-aa0e-fecb35515c19 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:12:12.339198 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd5a57e3-d2ea-4876-aa0e-fecb35515c19 admin admin] PATH_INFO: `/v3/users` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:12:12.340138 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5a57e3-d2ea-4876-aa0e-fecb35515c19 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:12:12.344863 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd5a57e3-d2ea-4876-aa0e-fecb35515c19 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:12:12.401661 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 816/1638] 10.222.0.52 () {60 vars in 1143 bytes} [Wed Jul 13 22:12:12 2022] POST /identity/v3/users => generated 283 bytes in 290 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 13 22:12:12.406086 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be52182-d7f4-42d6-90da-6f3d16dfc634 admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:12:12.418077 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7be52182-d7f4-42d6-90da-6f3d16dfc634 admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:12:12.429655 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d849ad-0693-44f0-b1e6-16284255dbb8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:12:12.431107 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d849ad-0693-44f0-b1e6-16284255dbb8 admin admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:12:12.431107 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d849ad-0693-44f0-b1e6-16284255dbb8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:12:12.431498 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95d849ad-0693-44f0-b1e6-16284255dbb8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:12:12.432255 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d849ad-0693-44f0-b1e6-16284255dbb8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:12:12.435142 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95d849ad-0693-44f0-b1e6-16284255dbb8 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:12:12.452314 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5a57e3-d2ea-4876-aa0e-fecb35515c19 admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:12:12.462465 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 823/1639] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jul 13 22:12:12 2022] GET /identity/v3/roles => generated 1420 bytes in 57 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 22:12:12.465825 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd5a57e3-d2ea-4876-aa0e-fecb35515c19 admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:12:12.486219 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e0d73b00-ab74-4fb2-8121-c6fe74ee2984 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:12:12.487743 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d73b00-ab74-4fb2-8121-c6fe74ee2984 admin admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:12:12.488158 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d73b00-ab74-4fb2-8121-c6fe74ee2984 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:12:12.488530 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e0d73b00-ab74-4fb2-8121-c6fe74ee2984 admin admin] PATH_INFO: `/v3/roles` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:12:12.493611 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d73b00-ab74-4fb2-8121-c6fe74ee2984 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:12:12.495486 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e0d73b00-ab74-4fb2-8121-c6fe74ee2984 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:12:12.515486 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 817/1640] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jul 13 22:12:12 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 22:12:12.517726 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d849ad-0693-44f0-b1e6-16284255dbb8 admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:12:12.534865 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95d849ad-0693-44f0-b1e6-16284255dbb8 admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:12:12.553194 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1849d801-8103-4532-b89c-9c6af57cf308 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:12:12.554558 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1849d801-8103-4532-b89c-9c6af57cf308 admin admin] REQUEST_METHOD: `PUT` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:12:12.554989 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1849d801-8103-4532-b89c-9c6af57cf308 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:12:12.555403 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1849d801-8103-4532-b89c-9c6af57cf308 admin admin] PATH_INFO: `/v3/projects/adbb20e4e1ab4195bde85315c5327343/users/f03cae641c7c417d8321874bfa8e1756/roles/2a5e86976394498abb6310697cdb3f8b` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:12:12.595581 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1849d801-8103-4532-b89c-9c6af57cf308 admin admin] RBAC: Authorizing `identity:create_grant(project_id=adbb20e4e1ab4195bde85315c5327343, user_id=f03cae641c7c417d8321874bfa8e1756, role_id=2a5e86976394498abb6310697cdb3f8b)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:12:12.597904 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1849d801-8103-4532-b89c-9c6af57cf308 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:12:12.651956 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 824/1641] 10.222.0.52 () {60 vars in 1482 bytes} [Wed Jul 13 22:12:12 2022] PUT /identity/v3/projects/adbb20e4e1ab4195bde85315c5327343/users/f03cae641c7c417d8321874bfa8e1756/roles/2a5e86976394498abb6310697cdb3f8b => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 22:12:12.653251 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06d602d-679d-49cd-977e-45de377a889b None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:12:12.653544 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06d602d-679d-49cd-977e-45de377a889b None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:12:12.653849 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e06d602d-679d-49cd-977e-45de377a889b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:12:12.709467 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-e06d602d-679d-49cd-977e-45de377a889b None None] MFA Rules not processed for user `f03cae641c7c417d8321874bfa8e1756`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 22:12:12.710776 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-e06d602d-679d-49cd-977e-45de377a889b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:12:12.802092 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1849d801-8103-4532-b89c-9c6af57cf308 admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:12:12.814221 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 818/1642] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jul 13 22:12:12 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 22:12:12.934250 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1849d801-8103-4532-b89c-9c6af57cf308 admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:12:12.940231 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537eff2d-9750-4dd7-931c-ad88ad2d3459 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:12:12.941018 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537eff2d-9750-4dd7-931c-ad88ad2d3459 admin admin] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:12:12.941464 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537eff2d-9750-4dd7-931c-ad88ad2d3459 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:12:12.941929 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-537eff2d-9750-4dd7-931c-ad88ad2d3459 admin admin] PATH_INFO: `/v3/projects` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:12:12.942780 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537eff2d-9750-4dd7-931c-ad88ad2d3459 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:12:12.944790 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-537eff2d-9750-4dd7-931c-ad88ad2d3459 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:12:13.005717 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 825/1643] 10.222.0.52 () {60 vars in 1152 bytes} [Wed Jul 13 22:12:12 2022] POST /identity/v3/projects => generated 365 bytes in 204 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 22:12:13.007046 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06d602d-679d-49cd-977e-45de377a889b None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:12:13.044744 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e06d602d-679d-49cd-977e-45de377a889b None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:12:13.252863 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff35268-aebb-4bbc-92ce-ee3add179dee admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:12:13.253761 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff35268-aebb-4bbc-92ce-ee3add179dee admin admin] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:12:13.254231 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff35268-aebb-4bbc-92ce-ee3add179dee admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:12:13.254613 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cff35268-aebb-4bbc-92ce-ee3add179dee admin admin] PATH_INFO: `/v3/users` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:12:13.256361 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff35268-aebb-4bbc-92ce-ee3add179dee admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:12:13.258977 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cff35268-aebb-4bbc-92ce-ee3add179dee admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:12:13.316866 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 819/1644] 10.222.0.52 () {60 vars in 1143 bytes} [Wed Jul 13 22:12:13 2022] POST /identity/v3/users => generated 289 bytes in 310 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 13 22:12:13.322930 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537eff2d-9750-4dd7-931c-ad88ad2d3459 admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:12:13.344746 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-537eff2d-9750-4dd7-931c-ad88ad2d3459 admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:12:13.425787 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d84425c-a168-4dfd-a1c8-330ff61270e5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:12:13.426558 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d84425c-a168-4dfd-a1c8-330ff61270e5 admin admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:12:13.428318 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d84425c-a168-4dfd-a1c8-330ff61270e5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:12:13.428785 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d84425c-a168-4dfd-a1c8-330ff61270e5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:12:13.429798 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d84425c-a168-4dfd-a1c8-330ff61270e5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:12:13.431906 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d84425c-a168-4dfd-a1c8-330ff61270e5 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:12:13.478943 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 826/1645] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jul 13 22:12:13 2022] GET /identity/v3/roles => generated 1420 bytes in 158 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 22:12:13.480070 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff35268-aebb-4bbc-92ce-ee3add179dee admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:12:13.501044 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cff35268-aebb-4bbc-92ce-ee3add179dee admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:12:13.525768 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffca0473-9dc3-4da4-843e-552a11538a06 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:12:13.528085 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffca0473-9dc3-4da4-843e-552a11538a06 admin admin] REQUEST_METHOD: `PUT` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:12:13.529688 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffca0473-9dc3-4da4-843e-552a11538a06 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:12:13.530089 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffca0473-9dc3-4da4-843e-552a11538a06 admin admin] PATH_INFO: `/v3/projects/a4bb5cfd427349f7a00f71f860b1e70d/users/d9d95fd44fb046bb9fec24b881b8ae0c/roles/44efba2579f342df8a0789c59d163ec3` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:12:13.589631 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffca0473-9dc3-4da4-843e-552a11538a06 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a4bb5cfd427349f7a00f71f860b1e70d, user_id=d9d95fd44fb046bb9fec24b881b8ae0c, role_id=44efba2579f342df8a0789c59d163ec3)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:12:13.593048 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffca0473-9dc3-4da4-843e-552a11538a06 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:12:13.634615 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 820/1646] 10.222.0.52 () {60 vars in 1482 bytes} [Wed Jul 13 22:12:13 2022] PUT /identity/v3/projects/a4bb5cfd427349f7a00f71f860b1e70d/users/d9d95fd44fb046bb9fec24b881b8ae0c/roles/44efba2579f342df8a0789c59d163ec3 => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 22:12:13.638970 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d84425c-a168-4dfd-a1c8-330ff61270e5 admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:12:13.651352 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:12:13.661922 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d84425c-a168-4dfd-a1c8-330ff61270e5 admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:12:13.759268 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a4d8f88-16fc-4d5d-bd29-c86d35379394 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:12:13.760952 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4d8f88-16fc-4d5d-bd29-c86d35379394 admin admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:12:13.761444 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4d8f88-16fc-4d5d-bd29-c86d35379394 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:12:13.761912 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a4d8f88-16fc-4d5d-bd29-c86d35379394 admin admin] PATH_INFO: `/v3/roles` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:12:13.763381 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a4d8f88-16fc-4d5d-bd29-c86d35379394 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:12:13.766513 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a4d8f88-16fc-4d5d-bd29-c86d35379394 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:12:13.785934 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 827/1647] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jul 13 22:12:13 2022] GET /identity/v3/roles => generated 1420 bytes in 148 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 22:12:13.790347 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffca0473-9dc3-4da4-843e-552a11538a06 admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:12:13.799120 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:12:13.804004 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffca0473-9dc3-4da4-843e-552a11538a06 admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:12:13.810812 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e8deb1-2adb-4c9f-a541-28327c2d3560 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:12:13.812874 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e8deb1-2adb-4c9f-a541-28327c2d3560 admin admin] REQUEST_METHOD: `PUT` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:12:13.813527 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e8deb1-2adb-4c9f-a541-28327c2d3560 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:12:13.814035 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-55e8deb1-2adb-4c9f-a541-28327c2d3560 admin admin] PATH_INFO: `/v3/projects/a4bb5cfd427349f7a00f71f860b1e70d/users/d9d95fd44fb046bb9fec24b881b8ae0c/roles/44efba2579f342df8a0789c59d163ec3` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:12:13.818445 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e8deb1-2adb-4c9f-a541-28327c2d3560 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a4bb5cfd427349f7a00f71f860b1e70d, user_id=d9d95fd44fb046bb9fec24b881b8ae0c, role_id=44efba2579f342df8a0789c59d163ec3)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:12:13.821603 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-55e8deb1-2adb-4c9f-a541-28327c2d3560 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:12:13.836159 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 821/1648] 10.222.0.52 () {60 vars in 1482 bytes} [Wed Jul 13 22:12:13 2022] PUT /identity/v3/projects/a4bb5cfd427349f7a00f71f860b1e70d/users/d9d95fd44fb046bb9fec24b881b8ae0c/roles/44efba2579f342df8a0789c59d163ec3 => generated 0 bytes in 46 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 22:12:13.848709 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717d4749-4f4a-40fe-9a46-9be05d2089db None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:12:13.848998 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717d4749-4f4a-40fe-9a46-9be05d2089db None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:12:13.849275 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-717d4749-4f4a-40fe-9a46-9be05d2089db None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:12:13.903975 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-717d4749-4f4a-40fe-9a46-9be05d2089db None None] MFA Rules not processed for user `d9d95fd44fb046bb9fec24b881b8ae0c`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 22:12:13.908300 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-717d4749-4f4a-40fe-9a46-9be05d2089db None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:12:14.079450 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 828/1649] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jul 13 22:12:13 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 232 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 22:12:14.097089 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e8deb1-2adb-4c9f-a541-28327c2d3560 admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:12:14.099092 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-55e8deb1-2adb-4c9f-a541-28327c2d3560 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:12:14.230279 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-55e8deb1-2adb-4c9f-a541-28327c2d3560 admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:12:14.243313 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef43558d-d2cb-477d-8648-d51471da9e25 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc77e1ea1ada4700b809beb1c980c5e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417e3e2f1333481795a089bef2e24bea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:12:14.244505 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef43558d-d2cb-477d-8648-d51471da9e25 service cinder] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:12:14.245388 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef43558d-d2cb-477d-8648-d51471da9e25 service cinder] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:12:14.245885 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef43558d-d2cb-477d-8648-d51471da9e25 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:12:14.287286 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef43558d-d2cb-477d-8648-d51471da9e25 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:12:14.289339 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef43558d-d2cb-477d-8648-d51471da9e25 service cinder] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:12:14.303910 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 822/1650] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jul 13 22:12:14 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 207 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 22:12:15.366865 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717d4749-4f4a-40fe-9a46-9be05d2089db None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:12:15.497308 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-717d4749-4f4a-40fe-9a46-9be05d2089db None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:12:15.509568 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5958f1d9-e69c-4202-af1f-25f541c29f60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:12:15.510511 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5958f1d9-e69c-4202-af1f-25f541c29f60 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:12:15.510815 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5958f1d9-e69c-4202-af1f-25f541c29f60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:12:15.511086 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5958f1d9-e69c-4202-af1f-25f541c29f60 admin admin] PATH_INFO: `/v3/users/f03cae641c7c417d8321874bfa8e1756` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:12:15.513166 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5958f1d9-e69c-4202-af1f-25f541c29f60 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f03cae641c7c417d8321874bfa8e1756)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:12:15.515187 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5958f1d9-e69c-4202-af1f-25f541c29f60 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:12:15.595391 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=95,keystone:DELETE=5 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:12:15.644215 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-5958f1d9-e69c-4202-af1f-25f541c29f60 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f03cae641c7c417d8321874bfa8e1756'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:12:15.669002 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-5958f1d9-e69c-4202-af1f-25f541c29f60 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f03cae641c7c417d8321874bfa8e1756'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:12:15.694398 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-5958f1d9-e69c-4202-af1f-25f541c29f60 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f03cae641c7c417d8321874bfa8e1756'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:12:15.695193 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-5958f1d9-e69c-4202-af1f-25f541c29f60 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f03cae641c7c417d8321874bfa8e1756'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:12:15.695805 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-5958f1d9-e69c-4202-af1f-25f541c29f60 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f03cae641c7c417d8321874bfa8e1756'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:12:15.720110 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 829/1651] 10.222.0.52 () {58 vars in 1223 bytes} [Wed Jul 13 22:12:15 2022] DELETE /identity/v3/users/f03cae641c7c417d8321874bfa8e1756 => generated 0 bytes in 354 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 22:12:15.725047 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22984ae0-48e0-4313-9443-1908ac008831 None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:12:15.725547 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22984ae0-48e0-4313-9443-1908ac008831 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:12:15.725547 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22984ae0-48e0-4313-9443-1908ac008831 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:12:15.744610 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-22984ae0-48e0-4313-9443-1908ac008831 None None] MFA Rules not processed for user `cc99c93da5074cf5a48bc2aa9a92c5e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 22:12:15.745830 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-22984ae0-48e0-4313-9443-1908ac008831 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:12:15.875212 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 823/1652] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jul 13 22:12:15 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 151 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 22:12:15.958452 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5958f1d9-e69c-4202-af1f-25f541c29f60 admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:12:15.960576 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-5958f1d9-e69c-4202-af1f-25f541c29f60 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:12:16.129439 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5958f1d9-e69c-4202-af1f-25f541c29f60 admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:12:16.135313 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b84d8362-4829-4eac-a727-4167c12f7daf service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '847ec9a1bd4d4dec804318a5c845895a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417e3e2f1333481795a089bef2e24bea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:12:16.136501 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84d8362-4829-4eac-a727-4167c12f7daf service neutron] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:12:16.136982 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84d8362-4829-4eac-a727-4167c12f7daf service neutron] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:12:16.137624 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b84d8362-4829-4eac-a727-4167c12f7daf service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:12:16.155882 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b84d8362-4829-4eac-a727-4167c12f7daf service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:12:16.158462 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b84d8362-4829-4eac-a727-4167c12f7daf service neutron] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:12:16.184351 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 830/1653] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jul 13 22:12:15 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 227 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 22:12:16.656166 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d88bfd0-d488-42ed-8fc9-00784ea98c2f None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:12:16.656445 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d88bfd0-d488-42ed-8fc9-00784ea98c2f None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:12:16.656723 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d88bfd0-d488-42ed-8fc9-00784ea98c2f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:12:16.732673 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-2d88bfd0-d488-42ed-8fc9-00784ea98c2f None None] MFA Rules not processed for user `99a16a550cdf407880fde2ba580fa0c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 22:12:16.734046 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-2d88bfd0-d488-42ed-8fc9-00784ea98c2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:12:16.899700 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 824/1654] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jul 13 22:12:16 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 244 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 22:12:16.912176 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ddf9aa-493b-4d00-9ad5-39da6121a793 None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:12:16.914035 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ddf9aa-493b-4d00-9ad5-39da6121a793 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:12:16.914468 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17ddf9aa-493b-4d00-9ad5-39da6121a793 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:12:16.942768 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-17ddf9aa-493b-4d00-9ad5-39da6121a793 None None] MFA Rules not processed for user `99a16a550cdf407880fde2ba580fa0c2`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 22:12:16.944204 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-17ddf9aa-493b-4d00-9ad5-39da6121a793 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:12:16.953379 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 831/1655] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jul 13 22:12:16 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 22:12:16.964399 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d88bfd0-d488-42ed-8fc9-00784ea98c2f None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:12:16.966353 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-2d88bfd0-d488-42ed-8fc9-00784ea98c2f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:12:17.008858 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:12:17.074583 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d88bfd0-d488-42ed-8fc9-00784ea98c2f None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:12:17.086472 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fcf89cdd-2c94-4017-91ef-c81fea3de639 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84281091687464990ffbca590b9343b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417e3e2f1333481795a089bef2e24bea', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:12:17.089286 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf89cdd-2c94-4017-91ef-c81fea3de639 service nova] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:12:17.089799 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf89cdd-2c94-4017-91ef-c81fea3de639 service nova] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:12:17.090465 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fcf89cdd-2c94-4017-91ef-c81fea3de639 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:12:17.111710 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf89cdd-2c94-4017-91ef-c81fea3de639 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:12:17.113824 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fcf89cdd-2c94-4017-91ef-c81fea3de639 service nova] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:12:17.136743 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 825/1656] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jul 13 22:12:16 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 173 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 22:12:18.071514 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ddf9aa-493b-4d00-9ad5-39da6121a793 None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:12:18.073951 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-17ddf9aa-493b-4d00-9ad5-39da6121a793 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:12:18.101565 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17ddf9aa-493b-4d00-9ad5-39da6121a793 None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:12:18.118567 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62bf06e7-5bed-4850-a86c-6ab15254be05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:12:18.120163 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bf06e7-5bed-4850-a86c-6ab15254be05 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:12:18.120974 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bf06e7-5bed-4850-a86c-6ab15254be05 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:12:18.121274 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-62bf06e7-5bed-4850-a86c-6ab15254be05 admin admin] PATH_INFO: `/v3/projects/adbb20e4e1ab4195bde85315c5327343` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:12:18.123724 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62bf06e7-5bed-4850-a86c-6ab15254be05 admin admin] RBAC: Authorizing `identity:delete_project(project_id=adbb20e4e1ab4195bde85315c5327343)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:12:18.130790 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-62bf06e7-5bed-4850-a86c-6ab15254be05 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:12:18.139061 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca36d155-1793-413d-9a9d-4d5876b9a987 None None] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:12:18.143669 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca36d155-1793-413d-9a9d-4d5876b9a987 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:12:18.144323 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca36d155-1793-413d-9a9d-4d5876b9a987 None None] PATH_INFO: `/` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:12:18.148213 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 826/1657] 10.222.0.52 () {54 vars in 912 bytes} [Wed Jul 13 22:12:18 2022] GET /identity => generated 270 bytes in 10 msecs (HTTP/1.1 300) 6 headers in 234 bytes (2 switches on core 0) Jul 13 22:12:18.154003 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02c19ec-8a6e-4c3f-906e-15419b6fbe31 None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:12:18.155790 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02c19ec-8a6e-4c3f-906e-15419b6fbe31 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:12:18.156286 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c02c19ec-8a6e-4c3f-906e-15419b6fbe31 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:12:18.264784 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-c02c19ec-8a6e-4c3f-906e-15419b6fbe31 None None] MFA Rules not processed for user `847ec9a1bd4d4dec804318a5c845895a`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 22:12:18.285507 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-c02c19ec-8a6e-4c3f-906e-15419b6fbe31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:12:18.328846 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-62bf06e7-5bed-4850-a86c-6ab15254be05 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'adbb20e4e1ab4195bde85315c5327343', 'request_id': 'req-62bf06e7-5bed-4850-a86c-6ab15254be05'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:12:18.358016 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-62bf06e7-5bed-4850-a86c-6ab15254be05 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'adbb20e4e1ab4195bde85315c5327343', 'request_id': 'req-62bf06e7-5bed-4850-a86c-6ab15254be05'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:12:18.385786 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-62bf06e7-5bed-4850-a86c-6ab15254be05 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'adbb20e4e1ab4195bde85315c5327343', 'request_id': 'req-62bf06e7-5bed-4850-a86c-6ab15254be05'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:12:18.423245 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-62bf06e7-5bed-4850-a86c-6ab15254be05 admin admin] The token cache is being invalidate because project adbb20e4e1ab4195bde85315c5327343 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80381) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 22:12:18.423807 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-62bf06e7-5bed-4850-a86c-6ab15254be05 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:12:18.428286 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-62bf06e7-5bed-4850-a86c-6ab15254be05 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:12:18.438557 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 832/1658] 10.222.0.52 () {58 vars in 1232 bytes} [Wed Jul 13 22:12:18 2022] DELETE /identity/v3/projects/adbb20e4e1ab4195bde85315c5327343 => generated 0 bytes in 367 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 22:12:18.443400 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62bf06e7-5bed-4850-a86c-6ab15254be05 admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:12:18.445258 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-62bf06e7-5bed-4850-a86c-6ab15254be05 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:12:18.506398 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 827/1659] 10.222.0.52 () {58 vars in 1009 bytes} [Wed Jul 13 22:12:18 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 354 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 13 22:12:18.524721 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=86,keystone:DELETE=12,keystone:INSERT=2 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:12:18.552150 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02c19ec-8a6e-4c3f-906e-15419b6fbe31 None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:12:18.552150 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-c02c19ec-8a6e-4c3f-906e-15419b6fbe31 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:12:18.577453 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c02c19ec-8a6e-4c3f-906e-15419b6fbe31 None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:12:18.589139 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6560230-0b5a-466e-8d01-58a51f79a86e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '847ec9a1bd4d4dec804318a5c845895a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417e3e2f1333481795a089bef2e24bea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:12:18.590207 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6560230-0b5a-466e-8d01-58a51f79a86e service neutron] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:12:18.590525 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6560230-0b5a-466e-8d01-58a51f79a86e service neutron] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:12:18.592377 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6560230-0b5a-466e-8d01-58a51f79a86e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:12:18.624245 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6560230-0b5a-466e-8d01-58a51f79a86e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:12:18.630472 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6560230-0b5a-466e-8d01-58a51f79a86e service neutron] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:12:18.656655 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-62bf06e7-5bed-4850-a86c-6ab15254be05 admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:12:18.660856 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 828/1660] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jul 13 22:12:18 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 13 22:12:18.670988 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d16a7b0-86af-4df7-8df8-758f704af9c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:12:18.671930 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d16a7b0-86af-4df7-8df8-758f704af9c5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:12:18.672211 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d16a7b0-86af-4df7-8df8-758f704af9c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:12:18.672691 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d16a7b0-86af-4df7-8df8-758f704af9c5 admin admin] PATH_INFO: `/v3/users/d9d95fd44fb046bb9fec24b881b8ae0c` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:12:18.674383 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d16a7b0-86af-4df7-8df8-758f704af9c5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=d9d95fd44fb046bb9fec24b881b8ae0c)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:12:18.679273 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d16a7b0-86af-4df7-8df8-758f704af9c5 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:12:18.808934 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-2d16a7b0-86af-4df7-8df8-758f704af9c5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'd9d95fd44fb046bb9fec24b881b8ae0c'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:12:18.828100 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-2d16a7b0-86af-4df7-8df8-758f704af9c5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'd9d95fd44fb046bb9fec24b881b8ae0c'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:12:18.853423 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-2d16a7b0-86af-4df7-8df8-758f704af9c5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'd9d95fd44fb046bb9fec24b881b8ae0c'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:12:18.855406 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-2d16a7b0-86af-4df7-8df8-758f704af9c5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'd9d95fd44fb046bb9fec24b881b8ae0c'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:12:18.856640 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-2d16a7b0-86af-4df7-8df8-758f704af9c5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'd9d95fd44fb046bb9fec24b881b8ae0c'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:12:18.893581 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 833/1661] 10.222.0.52 () {58 vars in 1223 bytes} [Wed Jul 13 22:12:18 2022] DELETE /identity/v3/users/d9d95fd44fb046bb9fec24b881b8ae0c => generated 0 bytes in 451 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 22:12:20.293177 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17a12d4-1b90-419e-8ab1-24ba32749684 None None] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:12:20.294715 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17a12d4-1b90-419e-8ab1-24ba32749684 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:12:20.295344 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c17a12d4-1b90-419e-8ab1-24ba32749684 None None] PATH_INFO: `/` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:12:20.297675 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 829/1662] 10.222.0.52 () {54 vars in 960 bytes} [Wed Jul 13 22:12:20 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 22:12:20.328831 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f82298b-3d9c-496a-8618-5d4931c1f2a2 None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:12:20.330541 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f82298b-3d9c-496a-8618-5d4931c1f2a2 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:12:20.330541 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f82298b-3d9c-496a-8618-5d4931c1f2a2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:12:20.384432 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-8f82298b-3d9c-496a-8618-5d4931c1f2a2 None None] MFA Rules not processed for user `ac673d641d8849c79bf4cf71c9805587`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 22:12:20.386952 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-8f82298b-3d9c-496a-8618-5d4931c1f2a2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:12:20.563623 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 834/1663] 10.222.0.52 () {58 vars in 1057 bytes} [Wed Jul 13 22:12:20 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 243 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 22:12:20.564080 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17a12d4-1b90-419e-8ab1-24ba32749684 None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:12:20.598800 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c17a12d4-1b90-419e-8ab1-24ba32749684 None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:12:20.629891 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27b173d-b1ef-4769-ad3a-7d798f6a3c17 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac673d641d8849c79bf4cf71c9805587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417e3e2f1333481795a089bef2e24bea', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:12:20.630867 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27b173d-b1ef-4769-ad3a-7d798f6a3c17 service placement] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:12:20.631299 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27b173d-b1ef-4769-ad3a-7d798f6a3c17 service placement] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:12:20.631701 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b27b173d-b1ef-4769-ad3a-7d798f6a3c17 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:12:20.634336 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-b27b173d-b1ef-4769-ad3a-7d798f6a3c17 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:12:20.692076 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27b173d-b1ef-4769-ad3a-7d798f6a3c17 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:12:20.699911 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b27b173d-b1ef-4769-ad3a-7d798f6a3c17 service placement] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:12:20.733744 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 830/1664] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jul 13 22:12:20 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 170 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 22:12:21.443750 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a98e046-16ec-44f6-808e-50105512ddd7 None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:12:21.444828 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a98e046-16ec-44f6-808e-50105512ddd7 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:12:21.445288 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a98e046-16ec-44f6-808e-50105512ddd7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:12:21.527605 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-4a98e046-16ec-44f6-808e-50105512ddd7 None None] MFA Rules not processed for user `83d2f82cf096424aa4979f443a2729d6`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 22:12:21.529032 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-4a98e046-16ec-44f6-808e-50105512ddd7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:12:21.663823 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:12:21.811949 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27b173d-b1ef-4769-ad3a-7d798f6a3c17 service placement] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:12:21.817182 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 835/1665] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jul 13 22:12:21 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 373 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 22:12:21.817342 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-b27b173d-b1ef-4769-ad3a-7d798f6a3c17 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:12:22.052719 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b27b173d-b1ef-4769-ad3a-7d798f6a3c17 service placement] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:12:22.064844 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ecee38-6855-4b3a-8a5c-cd16c0681d25 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84281091687464990ffbca590b9343b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417e3e2f1333481795a089bef2e24bea', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:12:22.067592 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ecee38-6855-4b3a-8a5c-cd16c0681d25 service nova] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:12:22.068444 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ecee38-6855-4b3a-8a5c-cd16c0681d25 service nova] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:12:22.069505 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00ecee38-6855-4b3a-8a5c-cd16c0681d25 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:12:22.093903 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ecee38-6855-4b3a-8a5c-cd16c0681d25 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:12:22.102931 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00ecee38-6855-4b3a-8a5c-cd16c0681d25 service nova] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:12:22.141009 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 831/1666] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jul 13 22:12:21 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 329 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 13 22:12:22.200710 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa61f319-399d-4198-acde-9af1282595e5 None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:12:22.200710 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa61f319-399d-4198-acde-9af1282595e5 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:12:22.201150 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa61f319-399d-4198-acde-9af1282595e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:12:22.266289 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-fa61f319-399d-4198-acde-9af1282595e5 None None] MFA Rules not processed for user `18b51e7024fc46a8af3644af27ab886f`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 22:12:22.267843 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-fa61f319-399d-4198-acde-9af1282595e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:12:22.450264 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ecee38-6855-4b3a-8a5c-cd16c0681d25 service nova] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:12:22.451815 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-00ecee38-6855-4b3a-8a5c-cd16c0681d25 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:12:22.462373 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:12:22.673948 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 836/1667] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jul 13 22:12:22 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 475 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 22:12:22.712151 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa61f319-399d-4198-acde-9af1282595e5 None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:12:22.743008 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa61f319-399d-4198-acde-9af1282595e5 None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:12:22.759934 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22192e2f-2856-46e5-9098-ed9e406adf6b service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84281091687464990ffbca590b9343b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417e3e2f1333481795a089bef2e24bea', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:12:22.760640 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22192e2f-2856-46e5-9098-ed9e406adf6b service nova] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:12:22.760919 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22192e2f-2856-46e5-9098-ed9e406adf6b service nova] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:12:22.761193 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22192e2f-2856-46e5-9098-ed9e406adf6b service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:12:22.786271 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22192e2f-2856-46e5-9098-ed9e406adf6b service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:12:22.788351 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22192e2f-2856-46e5-9098-ed9e406adf6b service nova] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:12:22.832890 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 837/1668] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jul 13 22:12:22 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 120 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 13 22:12:22.841520 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22192e2f-2856-46e5-9098-ed9e406adf6b service nova] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:12:22.845493 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-22192e2f-2856-46e5-9098-ed9e406adf6b service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:12:22.880250 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00ecee38-6855-4b3a-8a5c-cd16c0681d25 service nova] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:12:22.903063 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc1cf84-aab7-4102-9cd3-3cbb307cec97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:12:22.907021 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22192e2f-2856-46e5-9098-ed9e406adf6b service nova] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:12:22.912958 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc1cf84-aab7-4102-9cd3-3cbb307cec97 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:12:22.913242 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc1cf84-aab7-4102-9cd3-3cbb307cec97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:12:22.913515 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc1cf84-aab7-4102-9cd3-3cbb307cec97 admin admin] PATH_INFO: `/v3/projects/a4bb5cfd427349f7a00f71f860b1e70d` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:12:22.920385 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc1cf84-aab7-4102-9cd3-3cbb307cec97 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a4bb5cfd427349f7a00f71f860b1e70d)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:12:22.936796 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c193850-351e-48e9-b20f-dcc1635f53c9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:12:22.947058 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c193850-351e-48e9-b20f-dcc1635f53c9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:12:22.947316 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c193850-351e-48e9-b20f-dcc1635f53c9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:12:22.948255 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c193850-351e-48e9-b20f-dcc1635f53c9 admin admin] PATH_INFO: `/v3/users/4ca2967918434dab95ed42fac6bbd318` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:12:22.949696 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c193850-351e-48e9-b20f-dcc1635f53c9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=4ca2967918434dab95ed42fac6bbd318)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:12:22.951896 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c193850-351e-48e9-b20f-dcc1635f53c9 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:12:22.958710 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc1cf84-aab7-4102-9cd3-3cbb307cec97 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:12:23.132002 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-0bc1cf84-aab7-4102-9cd3-3cbb307cec97 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a4bb5cfd427349f7a00f71f860b1e70d', 'request_id': 'req-0bc1cf84-aab7-4102-9cd3-3cbb307cec97'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:12:23.149100 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-8c193850-351e-48e9-b20f-dcc1635f53c9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '4ca2967918434dab95ed42fac6bbd318'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:12:23.174646 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-8c193850-351e-48e9-b20f-dcc1635f53c9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '4ca2967918434dab95ed42fac6bbd318'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:12:23.176030 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-0bc1cf84-aab7-4102-9cd3-3cbb307cec97 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a4bb5cfd427349f7a00f71f860b1e70d', 'request_id': 'req-0bc1cf84-aab7-4102-9cd3-3cbb307cec97'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:12:23.206419 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-0bc1cf84-aab7-4102-9cd3-3cbb307cec97 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a4bb5cfd427349f7a00f71f860b1e70d', 'request_id': 'req-0bc1cf84-aab7-4102-9cd3-3cbb307cec97'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:12:23.215589 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-8c193850-351e-48e9-b20f-dcc1635f53c9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '4ca2967918434dab95ed42fac6bbd318'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:12:23.217447 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-8c193850-351e-48e9-b20f-dcc1635f53c9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '4ca2967918434dab95ed42fac6bbd318'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:12:23.221673 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-8c193850-351e-48e9-b20f-dcc1635f53c9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '4ca2967918434dab95ed42fac6bbd318'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:12:23.251516 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-0bc1cf84-aab7-4102-9cd3-3cbb307cec97 admin admin] The token cache is being invalidate because project a4bb5cfd427349f7a00f71f860b1e70d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80382) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 22:12:23.251808 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-0bc1cf84-aab7-4102-9cd3-3cbb307cec97 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:12:23.259242 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-0bc1cf84-aab7-4102-9cd3-3cbb307cec97 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:12:23.265364 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 832/1669] 10.222.0.52 () {58 vars in 1232 bytes} [Wed Jul 13 22:12:22 2022] DELETE /identity/v3/projects/a4bb5cfd427349f7a00f71f860b1e70d => generated 0 bytes in 816 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 13 22:12:23.269784 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 838/1670] 10.222.0.52 () {58 vars in 1223 bytes} [Wed Jul 13 22:12:22 2022] DELETE /identity/v3/users/4ca2967918434dab95ed42fac6bbd318 => generated 0 bytes in 436 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 22:12:23.270612 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f16e401-c56f-4d81-be63-81d151c9176c None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:12:23.271035 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f16e401-c56f-4d81-be63-81d151c9176c None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:12:23.271315 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f16e401-c56f-4d81-be63-81d151c9176c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:12:23.274139 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014d9fb0-c888-430c-9cd4-b90bd043326c None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:12:23.274642 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014d9fb0-c888-430c-9cd4-b90bd043326c None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:12:23.275260 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-014d9fb0-c888-430c-9cd4-b90bd043326c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:12:23.338269 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-9f16e401-c56f-4d81-be63-81d151c9176c None None] MFA Rules not processed for user `cc99c93da5074cf5a48bc2aa9a92c5e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 22:12:23.343528 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-014d9fb0-c888-430c-9cd4-b90bd043326c None None] MFA Rules not processed for user `cc99c93da5074cf5a48bc2aa9a92c5e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 22:12:23.347641 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-9f16e401-c56f-4d81-be63-81d151c9176c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:12:23.350711 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-014d9fb0-c888-430c-9cd4-b90bd043326c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:12:23.401553 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:12:23.515141 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 839/1671] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jul 13 22:12:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 242 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 22:12:23.603093 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014d9fb0-c888-430c-9cd4-b90bd043326c None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:12:23.605307 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-014d9fb0-c888-430c-9cd4-b90bd043326c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:12:23.664527 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 833/1672] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jul 13 22:12:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 394 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 22:12:23.669671 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cac381-5fee-44f4-ba41-f937373a251a None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:12:23.669954 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cac381-5fee-44f4-ba41-f937373a251a None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:12:23.672189 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-98cac381-5fee-44f4-ba41-f937373a251a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:12:23.741573 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-98cac381-5fee-44f4-ba41-f937373a251a None None] MFA Rules not processed for user `cc99c93da5074cf5a48bc2aa9a92c5e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 22:12:23.744461 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-98cac381-5fee-44f4-ba41-f937373a251a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:12:23.780061 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-014d9fb0-c888-430c-9cd4-b90bd043326c None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:12:23.785066 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 834/1673] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jul 13 22:12:23 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 115 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 22:12:23.787318 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98cac381-5fee-44f4-ba41-f937373a251a None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:12:23.789173 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18c787d-8a42-466d-9ad9-c52dbcd01ffa service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '847ec9a1bd4d4dec804318a5c845895a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417e3e2f1333481795a089bef2e24bea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:12:23.791461 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18c787d-8a42-466d-9ad9-c52dbcd01ffa service neutron] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:12:23.793493 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18c787d-8a42-466d-9ad9-c52dbcd01ffa service neutron] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:12:23.793871 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18c787d-8a42-466d-9ad9-c52dbcd01ffa service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:12:23.820728 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-98cac381-5fee-44f4-ba41-f937373a251a None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:12:23.827450 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e24884-c9a1-4a57-b96d-8aec895ae788 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:12:23.828222 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e24884-c9a1-4a57-b96d-8aec895ae788 admin admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:12:23.828574 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e24884-c9a1-4a57-b96d-8aec895ae788 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:12:23.828683 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a18c787d-8a42-466d-9ad9-c52dbcd01ffa service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:12:23.828974 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e24884-c9a1-4a57-b96d-8aec895ae788 admin admin] PATH_INFO: `/v3/domains` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:12:23.830299 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e24884-c9a1-4a57-b96d-8aec895ae788 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 22:12:23.830714 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e24884-c9a1-4a57-b96d-8aec895ae788 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:12:23.832963 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e24884-c9a1-4a57-b96d-8aec895ae788 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:12:23.835002 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a18c787d-8a42-466d-9ad9-c52dbcd01ffa service neutron] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:12:23.869254 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 835/1674] 10.222.0.52 () {58 vars in 1152 bytes} [Wed Jul 13 22:12:23 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 22:12:23.874457 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e24884-c9a1-4a57-b96d-8aec895ae788 admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:12:23.884215 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 840/1675] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jul 13 22:12:23 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 282 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 13 22:12:23.895832 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e24884-c9a1-4a57-b96d-8aec895ae788 admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:12:23.938064 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a0c622-1c19-4d78-bebe-78f75a072309 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:12:23.939072 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a0c622-1c19-4d78-bebe-78f75a072309 admin admin] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:12:23.940954 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a0c622-1c19-4d78-bebe-78f75a072309 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:12:23.940954 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09a0c622-1c19-4d78-bebe-78f75a072309 admin admin] PATH_INFO: `/v3/projects` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:12:23.940954 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a0c622-1c19-4d78-bebe-78f75a072309 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:12:23.947136 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09a0c622-1c19-4d78-bebe-78f75a072309 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:12:23.980429 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:12:24.018730 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 836/1676] 10.222.0.52 () {60 vars in 1152 bytes} [Wed Jul 13 22:12:23 2022] POST /identity/v3/projects => generated 365 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 22:12:24.020047 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18c787d-8a42-466d-9ad9-c52dbcd01ffa service neutron] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:12:24.041613 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18c787d-8a42-466d-9ad9-c52dbcd01ffa service neutron] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:12:24.197186 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242237a0-dbce-4dd5-a09b-cee17466aa1c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:12:24.198276 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242237a0-dbce-4dd5-a09b-cee17466aa1c admin admin] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:12:24.198604 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242237a0-dbce-4dd5-a09b-cee17466aa1c admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:12:24.198861 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-242237a0-dbce-4dd5-a09b-cee17466aa1c admin admin] PATH_INFO: `/v3/users` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:12:24.199583 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-242237a0-dbce-4dd5-a09b-cee17466aa1c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:12:24.201695 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-242237a0-dbce-4dd5-a09b-cee17466aa1c admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:12:24.248134 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 841/1677] 10.222.0.52 () {60 vars in 1143 bytes} [Wed Jul 13 22:12:24 2022] POST /identity/v3/users => generated 283 bytes in 228 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 22:12:24.248548 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a0c622-1c19-4d78-bebe-78f75a072309 admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:12:24.263019 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09a0c622-1c19-4d78-bebe-78f75a072309 admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:12:24.276503 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d967e3-dc8e-4641-80cb-73a6ce8241b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:12:24.277423 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d967e3-dc8e-4641-80cb-73a6ce8241b0 admin admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:12:24.277882 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d967e3-dc8e-4641-80cb-73a6ce8241b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:12:24.278333 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e2d967e3-dc8e-4641-80cb-73a6ce8241b0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:12:24.279317 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d967e3-dc8e-4641-80cb-73a6ce8241b0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:12:24.281330 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e2d967e3-dc8e-4641-80cb-73a6ce8241b0 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:12:24.330549 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 837/1678] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jul 13 22:12:24 2022] GET /identity/v3/roles => generated 1420 bytes in 83 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 13 22:12:24.334098 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242237a0-dbce-4dd5-a09b-cee17466aa1c admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:12:24.349322 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-242237a0-dbce-4dd5-a09b-cee17466aa1c admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:12:24.368789 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f71bcf86-26ad-44ee-9a2d-7e0729946bb1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:12:24.369733 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71bcf86-26ad-44ee-9a2d-7e0729946bb1 admin admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:12:24.369839 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71bcf86-26ad-44ee-9a2d-7e0729946bb1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:12:24.370065 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f71bcf86-26ad-44ee-9a2d-7e0729946bb1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:12:24.370818 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f71bcf86-26ad-44ee-9a2d-7e0729946bb1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:12:24.404069 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f71bcf86-26ad-44ee-9a2d-7e0729946bb1 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:12:24.448201 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 842/1679] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jul 13 22:12:24 2022] GET /identity/v3/roles => generated 1420 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 22:12:24.452924 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d967e3-dc8e-4641-80cb-73a6ce8241b0 admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:12:24.473362 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e2d967e3-dc8e-4641-80cb-73a6ce8241b0 admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:12:24.494266 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd3965c-4ca8-45a3-9977-b7a89fd01b05 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:12:24.495296 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd3965c-4ca8-45a3-9977-b7a89fd01b05 admin admin] REQUEST_METHOD: `PUT` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:12:24.495608 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd3965c-4ca8-45a3-9977-b7a89fd01b05 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:12:24.495885 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fd3965c-4ca8-45a3-9977-b7a89fd01b05 admin admin] PATH_INFO: `/v3/projects/1396528f2e424a5caa45211fed5f48e2/users/340aab38ef56488c91b360983fc3a209/roles/2a5e86976394498abb6310697cdb3f8b` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:12:24.544366 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd3965c-4ca8-45a3-9977-b7a89fd01b05 admin admin] RBAC: Authorizing `identity:create_grant(project_id=1396528f2e424a5caa45211fed5f48e2, user_id=340aab38ef56488c91b360983fc3a209, role_id=2a5e86976394498abb6310697cdb3f8b)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:12:24.547147 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fd3965c-4ca8-45a3-9977-b7a89fd01b05 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:12:24.609819 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 838/1680] 10.222.0.52 () {60 vars in 1482 bytes} [Wed Jul 13 22:12:24 2022] PUT /identity/v3/projects/1396528f2e424a5caa45211fed5f48e2/users/340aab38ef56488c91b360983fc3a209/roles/2a5e86976394498abb6310697cdb3f8b => generated 0 bytes in 147 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 22:12:24.612085 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35481dad-dd2d-4163-9ccd-b1e7d50028b9 None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:12:24.612383 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35481dad-dd2d-4163-9ccd-b1e7d50028b9 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:12:24.612664 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35481dad-dd2d-4163-9ccd-b1e7d50028b9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:12:24.680776 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-35481dad-dd2d-4163-9ccd-b1e7d50028b9 None None] MFA Rules not processed for user `340aab38ef56488c91b360983fc3a209`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 22:12:24.682044 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-35481dad-dd2d-4163-9ccd-b1e7d50028b9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:12:24.801725 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:12:24.939850 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 843/1681] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jul 13 22:12:24 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 328 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 13 22:12:24.943845 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd3965c-4ca8-45a3-9977-b7a89fd01b05 admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:12:25.116193 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fd3965c-4ca8-45a3-9977-b7a89fd01b05 admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:12:25.122295 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33bad9a4-41c5-4b74-849e-2f55e2cb92a1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:12:25.123096 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bad9a4-41c5-4b74-849e-2f55e2cb92a1 admin admin] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:12:25.123612 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bad9a4-41c5-4b74-849e-2f55e2cb92a1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:12:25.124110 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33bad9a4-41c5-4b74-849e-2f55e2cb92a1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:12:25.125428 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33bad9a4-41c5-4b74-849e-2f55e2cb92a1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:12:25.128809 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33bad9a4-41c5-4b74-849e-2f55e2cb92a1 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:12:25.134091 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5e89c6-44f4-4202-9ee5-b107ec175064 None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:12:25.134735 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5e89c6-44f4-4202-9ee5-b107ec175064 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:12:25.135172 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1f5e89c6-44f4-4202-9ee5-b107ec175064 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:12:25.173250 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 839/1682] 10.222.0.52 () {60 vars in 1152 bytes} [Wed Jul 13 22:12:24 2022] POST /identity/v3/projects => generated 369 bytes in 230 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 22:12:25.186369 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33bad9a4-41c5-4b74-849e-2f55e2cb92a1 admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:12:25.243949 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-1f5e89c6-44f4-4202-9ee5-b107ec175064 None None] MFA Rules not processed for user `b866bacdafae4db6a32714c07c56515f`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 22:12:25.244859 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-1f5e89c6-44f4-4202-9ee5-b107ec175064 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:12:25.339092 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33bad9a4-41c5-4b74-849e-2f55e2cb92a1 admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:12:25.352353 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8a71b3-5a20-4b0d-bf6b-0f2b76a0f22e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:12:25.353286 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8a71b3-5a20-4b0d-bf6b-0f2b76a0f22e admin admin] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:12:25.355077 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8a71b3-5a20-4b0d-bf6b-0f2b76a0f22e admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:12:25.355510 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc8a71b3-5a20-4b0d-bf6b-0f2b76a0f22e admin admin] PATH_INFO: `/v3/users` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:12:25.359387 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc8a71b3-5a20-4b0d-bf6b-0f2b76a0f22e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:12:25.361517 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc8a71b3-5a20-4b0d-bf6b-0f2b76a0f22e admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:12:25.407297 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 840/1683] 10.222.0.52 () {60 vars in 1143 bytes} [Wed Jul 13 22:12:25 2022] POST /identity/v3/users => generated 291 bytes in 225 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 22:12:25.415032 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8a71b3-5a20-4b0d-bf6b-0f2b76a0f22e admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:12:25.436422 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc8a71b3-5a20-4b0d-bf6b-0f2b76a0f22e admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:12:25.440567 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 844/1684] 10.222.0.52 () {58 vars in 1009 bytes} [Wed Jul 13 22:12:25 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 307 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 13 22:12:25.451571 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5b1c74-14f4-4013-ab4f-771a089a8e26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:12:25.452555 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5b1c74-14f4-4013-ab4f-771a089a8e26 admin admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:12:25.452952 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5b1c74-14f4-4013-ab4f-771a089a8e26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:12:25.453041 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5b1c74-14f4-4013-ab4f-771a089a8e26 admin admin] PATH_INFO: `/v3/roles` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:12:25.453746 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be5b1c74-14f4-4013-ab4f-771a089a8e26 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:12:25.461022 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be5b1c74-14f4-4013-ab4f-771a089a8e26 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:12:25.486811 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 841/1685] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jul 13 22:12:25 2022] GET /identity/v3/roles => generated 1420 bytes in 70 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 22:12:25.488106 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5e89c6-44f4-4202-9ee5-b107ec175064 None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:12:25.513698 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1f5e89c6-44f4-4202-9ee5-b107ec175064 None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:12:25.529200 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954bd72e-299f-4481-9baa-0b2a4eaee805 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:12:25.530149 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954bd72e-299f-4481-9baa-0b2a4eaee805 admin admin] REQUEST_METHOD: `PUT` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:12:25.530429 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954bd72e-299f-4481-9baa-0b2a4eaee805 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:12:25.530962 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-954bd72e-299f-4481-9baa-0b2a4eaee805 admin admin] PATH_INFO: `/v3/projects/3ada8d5f66144266b8218959ebf1432b/users/8fc94ac51e584ef488d695cccd8a5690/roles/44efba2579f342df8a0789c59d163ec3` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:12:25.564456 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954bd72e-299f-4481-9baa-0b2a4eaee805 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3ada8d5f66144266b8218959ebf1432b, user_id=8fc94ac51e584ef488d695cccd8a5690, role_id=44efba2579f342df8a0789c59d163ec3)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:12:25.568507 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-954bd72e-299f-4481-9baa-0b2a4eaee805 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:12:25.611009 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 845/1686] 10.222.0.52 () {60 vars in 1482 bytes} [Wed Jul 13 22:12:25 2022] PUT /identity/v3/projects/3ada8d5f66144266b8218959ebf1432b/users/8fc94ac51e584ef488d695cccd8a5690/roles/44efba2579f342df8a0789c59d163ec3 => generated 0 bytes in 123 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 22:12:25.615120 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5b1c74-14f4-4013-ab4f-771a089a8e26 admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:12:25.627533 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5b1c74-14f4-4013-ab4f-771a089a8e26 admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:12:25.661633 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:INSERT=6,keystone:SELECT=94 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:12:25.725424 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954bd72e-299f-4481-9baa-0b2a4eaee805 admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:12:25.735487 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-954bd72e-299f-4481-9baa-0b2a4eaee805 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:12:25.770477 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1fafd1-a40b-4394-8e62-e81604abba21 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:12:25.771932 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1fafd1-a40b-4394-8e62-e81604abba21 admin admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:12:25.773937 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1fafd1-a40b-4394-8e62-e81604abba21 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:12:25.774249 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de1fafd1-a40b-4394-8e62-e81604abba21 admin admin] PATH_INFO: `/v3/roles` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:12:25.777347 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1fafd1-a40b-4394-8e62-e81604abba21 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:12:25.780486 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de1fafd1-a40b-4394-8e62-e81604abba21 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:12:25.815488 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 842/1687] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jul 13 22:12:25 2022] GET /identity/v3/roles => generated 1420 bytes in 200 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 22:12:25.816486 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1fafd1-a40b-4394-8e62-e81604abba21 admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:12:25.822869 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-de1fafd1-a40b-4394-8e62-e81604abba21 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:12:25.914423 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-954bd72e-299f-4481-9baa-0b2a4eaee805 admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:12:25.928687 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa8e8e4-2c0b-4518-9395-80633509aee2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:12:25.929608 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa8e8e4-2c0b-4518-9395-80633509aee2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:12:25.929891 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa8e8e4-2c0b-4518-9395-80633509aee2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:12:25.930166 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-efa8e8e4-2c0b-4518-9395-80633509aee2 admin admin] PATH_INFO: `/v3/users/7a993519e77740f5bd08fd11906ba50b` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:12:25.938406 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa8e8e4-2c0b-4518-9395-80633509aee2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7a993519e77740f5bd08fd11906ba50b)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:12:25.944496 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-efa8e8e4-2c0b-4518-9395-80633509aee2 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:12:26.076388 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=97,keystone:INSERT=1,keystone:DELETE=2 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:12:26.165974 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-efa8e8e4-2c0b-4518-9395-80633509aee2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7a993519e77740f5bd08fd11906ba50b'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:12:26.184713 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-efa8e8e4-2c0b-4518-9395-80633509aee2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7a993519e77740f5bd08fd11906ba50b'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:12:26.210811 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de1fafd1-a40b-4394-8e62-e81604abba21 admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:12:26.221182 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-efa8e8e4-2c0b-4518-9395-80633509aee2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7a993519e77740f5bd08fd11906ba50b'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:12:26.222221 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-efa8e8e4-2c0b-4518-9395-80633509aee2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7a993519e77740f5bd08fd11906ba50b'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:12:26.224003 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-efa8e8e4-2c0b-4518-9395-80633509aee2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7a993519e77740f5bd08fd11906ba50b'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:12:26.225677 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-09542fc9-fcd4-4865-818b-55f47619aa40 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eb1525233ba48c99b15d4cb75db4822', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417e3e2f1333481795a089bef2e24bea', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:12:26.227347 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09542fc9-fcd4-4865-818b-55f47619aa40 service swift] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:12:26.231531 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09542fc9-fcd4-4865-818b-55f47619aa40 service swift] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:12:26.231882 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-09542fc9-fcd4-4865-818b-55f47619aa40 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:12:26.251676 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-09542fc9-fcd4-4865-818b-55f47619aa40 service swift] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:12:26.275712 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 846/1688] 10.222.0.52 () {58 vars in 1223 bytes} [Wed Jul 13 22:12:25 2022] DELETE /identity/v3/users/7a993519e77740f5bd08fd11906ba50b => generated 0 bytes in 549 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 22:12:26.277110 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa8e8e4-2c0b-4518-9395-80633509aee2 admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:12:26.280616 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-efa8e8e4-2c0b-4518-9395-80633509aee2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:12:26.461471 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09542fc9-fcd4-4865-818b-55f47619aa40 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:12:26.468611 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-09542fc9-fcd4-4865-818b-55f47619aa40 service swift] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:12:26.503904 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 843/1689] 10.222.0.52 () {60 vars in 1384 bytes} [Wed Jul 13 22:12:25 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 687 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 13 22:12:26.515221 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e394d396-e02e-43cb-94f5-4be69b7faf1b None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:12:26.515511 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e394d396-e02e-43cb-94f5-4be69b7faf1b None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:12:26.515786 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e394d396-e02e-43cb-94f5-4be69b7faf1b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:12:26.521856 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-efa8e8e4-2c0b-4518-9395-80633509aee2 admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:12:26.544587 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf5b5bf-7f35-4522-a093-25c76a795970 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:12:26.546156 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf5b5bf-7f35-4522-a093-25c76a795970 admin admin] REQUEST_METHOD: `PUT` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:12:26.546911 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf5b5bf-7f35-4522-a093-25c76a795970 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:12:26.554248 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf5b5bf-7f35-4522-a093-25c76a795970 admin admin] PATH_INFO: `/v3/projects/3ada8d5f66144266b8218959ebf1432b/users/8fc94ac51e584ef488d695cccd8a5690/roles/44efba2579f342df8a0789c59d163ec3` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:12:26.560091 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf5b5bf-7f35-4522-a093-25c76a795970 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3ada8d5f66144266b8218959ebf1432b, user_id=8fc94ac51e584ef488d695cccd8a5690, role_id=44efba2579f342df8a0789c59d163ec3)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:12:26.565135 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf5b5bf-7f35-4522-a093-25c76a795970 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:12:26.572076 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-e394d396-e02e-43cb-94f5-4be69b7faf1b None None] MFA Rules not processed for user `cc99c93da5074cf5a48bc2aa9a92c5e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 22:12:26.577292 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-e394d396-e02e-43cb-94f5-4be69b7faf1b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:12:26.599590 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 847/1690] 10.222.0.52 () {60 vars in 1482 bytes} [Wed Jul 13 22:12:26 2022] PUT /identity/v3/projects/3ada8d5f66144266b8218959ebf1432b/users/8fc94ac51e584ef488d695cccd8a5690/roles/44efba2579f342df8a0789c59d163ec3 => generated 0 bytes in 323 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 22:12:26.600137 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf5b5bf-7f35-4522-a093-25c76a795970 admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:12:26.608229 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-3cf5b5bf-7f35-4522-a093-25c76a795970 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:12:26.684195 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:12:26.746423 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf5b5bf-7f35-4522-a093-25c76a795970 admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:12:26.756742 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ef3c11-f1b0-4e49-a63e-f399ea3141b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:12:26.757909 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ef3c11-f1b0-4e49-a63e-f399ea3141b8 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:12:26.758526 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ef3c11-f1b0-4e49-a63e-f399ea3141b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:12:26.759239 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-02ef3c11-f1b0-4e49-a63e-f399ea3141b8 admin admin] PATH_INFO: `/v3/projects/147ec3e62412490a9a8768973960f323` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:12:26.763740 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ef3c11-f1b0-4e49-a63e-f399ea3141b8 admin admin] RBAC: Authorizing `identity:delete_project(project_id=147ec3e62412490a9a8768973960f323)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:12:26.766747 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 844/1691] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jul 13 22:12:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 262 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 22:12:26.768612 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af97f2e9-ea5b-4d5e-8179-6a60c7e70e5d None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:12:26.769286 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af97f2e9-ea5b-4d5e-8179-6a60c7e70e5d None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:12:26.769723 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af97f2e9-ea5b-4d5e-8179-6a60c7e70e5d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:12:26.776644 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-02ef3c11-f1b0-4e49-a63e-f399ea3141b8 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:12:26.881204 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-af97f2e9-ea5b-4d5e-8179-6a60c7e70e5d None None] MFA Rules not processed for user `8fc94ac51e584ef488d695cccd8a5690`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 22:12:26.884106 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-af97f2e9-ea5b-4d5e-8179-6a60c7e70e5d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:12:26.903823 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:DELETE=10,keystone:SELECT=88,keystone:INSERT=2 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:12:26.941261 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-02ef3c11-f1b0-4e49-a63e-f399ea3141b8 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '147ec3e62412490a9a8768973960f323', 'request_id': 'req-02ef3c11-f1b0-4e49-a63e-f399ea3141b8'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:12:26.968960 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-02ef3c11-f1b0-4e49-a63e-f399ea3141b8 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '147ec3e62412490a9a8768973960f323', 'request_id': 'req-02ef3c11-f1b0-4e49-a63e-f399ea3141b8'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:12:26.987851 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-02ef3c11-f1b0-4e49-a63e-f399ea3141b8 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '147ec3e62412490a9a8768973960f323', 'request_id': 'req-02ef3c11-f1b0-4e49-a63e-f399ea3141b8'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:12:27.016979 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-02ef3c11-f1b0-4e49-a63e-f399ea3141b8 admin admin] The token cache is being invalidate because project 147ec3e62412490a9a8768973960f323 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80381) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 22:12:27.018007 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-02ef3c11-f1b0-4e49-a63e-f399ea3141b8 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:12:27.019715 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-02ef3c11-f1b0-4e49-a63e-f399ea3141b8 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:12:27.025477 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 848/1692] 10.222.0.52 () {58 vars in 1232 bytes} [Wed Jul 13 22:12:26 2022] DELETE /identity/v3/projects/147ec3e62412490a9a8768973960f323 => generated 0 bytes in 425 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 22:12:27.026924 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ef3c11-f1b0-4e49-a63e-f399ea3141b8 admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:12:27.035599 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-02ef3c11-f1b0-4e49-a63e-f399ea3141b8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:12:27.144862 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 845/1693] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jul 13 22:12:26 2022] POST /identity/v3/auth/tokens => generated 3303 bytes in 377 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 22:12:27.160238 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bc87d4-f2dc-4e57-bdf8-ea8b7a0f3891 None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:12:27.161076 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bc87d4-f2dc-4e57-bdf8-ea8b7a0f3891 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:12:27.162370 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12bc87d4-f2dc-4e57-bdf8-ea8b7a0f3891 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:12:27.223674 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-12bc87d4-f2dc-4e57-bdf8-ea8b7a0f3891 None None] MFA Rules not processed for user `cc99c93da5074cf5a48bc2aa9a92c5e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 22:12:27.228852 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-12bc87d4-f2dc-4e57-bdf8-ea8b7a0f3891 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:12:27.248050 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-02ef3c11-f1b0-4e49-a63e-f399ea3141b8 admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:12:27.263739 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c2ca1ea-c55b-4eb5-91ce-d7031dd0c3f5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '847ec9a1bd4d4dec804318a5c845895a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417e3e2f1333481795a089bef2e24bea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:12:27.265405 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2ca1ea-c55b-4eb5-91ce-d7031dd0c3f5 service neutron] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:12:27.267387 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2ca1ea-c55b-4eb5-91ce-d7031dd0c3f5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:12:27.267928 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c2ca1ea-c55b-4eb5-91ce-d7031dd0c3f5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:12:27.277693 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-7c2ca1ea-c55b-4eb5-91ce-d7031dd0c3f5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:12:27.395729 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 846/1694] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jul 13 22:12:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 251 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 22:12:27.397387 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12bc87d4-f2dc-4e57-bdf8-ea8b7a0f3891 None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:12:27.405572 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-12bc87d4-f2dc-4e57-bdf8-ea8b7a0f3891 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:12:27.452569 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c2ca1ea-c55b-4eb5-91ce-d7031dd0c3f5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:12:27.454023 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c2ca1ea-c55b-4eb5-91ce-d7031dd0c3f5 service neutron] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:12:27.497402 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12bc87d4-f2dc-4e57-bdf8-ea8b7a0f3891 None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:12:27.501034 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 849/1695] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jul 13 22:12:27 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 474 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 22:12:27.503623 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4d2e26-9a5a-4efc-af9d-082b64bfd1dc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc77e1ea1ada4700b809beb1c980c5e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417e3e2f1333481795a089bef2e24bea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:12:27.504820 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4d2e26-9a5a-4efc-af9d-082b64bfd1dc service cinder] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:12:27.505103 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4d2e26-9a5a-4efc-af9d-082b64bfd1dc service cinder] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:12:27.505389 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb3f166-0d0e-4254-94a2-e5348a112089 None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:12:27.505619 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3e4d2e26-9a5a-4efc-af9d-082b64bfd1dc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:12:27.505830 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb3f166-0d0e-4254-94a2-e5348a112089 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:12:27.506088 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bb3f166-0d0e-4254-94a2-e5348a112089 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:12:27.518232 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4d2e26-9a5a-4efc-af9d-082b64bfd1dc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:12:27.520137 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3e4d2e26-9a5a-4efc-af9d-082b64bfd1dc service cinder] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:12:27.549989 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 847/1696] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jul 13 22:12:27 2022] GET /identity/v3/auth/tokens => generated 3303 bytes in 151 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 22:12:27.595729 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-7bb3f166-0d0e-4254-94a2-e5348a112089 None None] MFA Rules not processed for user `cc99c93da5074cf5a48bc2aa9a92c5e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 22:12:27.598238 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-7bb3f166-0d0e-4254-94a2-e5348a112089 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:12:27.633101 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 850/1697] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jul 13 22:12:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 132 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 22:12:27.643155 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4d2e26-9a5a-4efc-af9d-082b64bfd1dc service cinder] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:12:27.660461 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3e4d2e26-9a5a-4efc-af9d-082b64bfd1dc service cinder] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:12:27.676109 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527ec3a6-0d20-409c-b7c7-9e70fbf83aae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:12:27.677143 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527ec3a6-0d20-409c-b7c7-9e70fbf83aae admin admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:12:27.677711 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527ec3a6-0d20-409c-b7c7-9e70fbf83aae admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:12:27.678214 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-527ec3a6-0d20-409c-b7c7-9e70fbf83aae admin admin] PATH_INFO: `/v3/domains` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:12:27.679230 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527ec3a6-0d20-409c-b7c7-9e70fbf83aae admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 22:12:27.679763 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527ec3a6-0d20-409c-b7c7-9e70fbf83aae admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:12:27.696764 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-527ec3a6-0d20-409c-b7c7-9e70fbf83aae admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:12:27.703094 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:12:27.729576 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 848/1698] 10.222.0.52 () {58 vars in 1152 bytes} [Wed Jul 13 22:12:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 22:12:27.732311 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb3f166-0d0e-4254-94a2-e5348a112089 None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:12:27.755795 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bb3f166-0d0e-4254-94a2-e5348a112089 None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:12:27.767880 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a94dbb2-e785-4880-a2cc-1f209171b07b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:12:27.769650 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a94dbb2-e785-4880-a2cc-1f209171b07b admin admin] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:12:27.769650 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a94dbb2-e785-4880-a2cc-1f209171b07b admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:12:27.769650 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a94dbb2-e785-4880-a2cc-1f209171b07b admin admin] PATH_INFO: `/v3/projects` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:12:27.769650 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a94dbb2-e785-4880-a2cc-1f209171b07b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:12:27.779517 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2a94dbb2-e785-4880-a2cc-1f209171b07b admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:12:27.878105 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 851/1699] 10.222.0.52 () {60 vars in 1152 bytes} [Wed Jul 13 22:12:27 2022] POST /identity/v3/projects => generated 381 bytes in 146 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 22:12:27.878371 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527ec3a6-0d20-409c-b7c7-9e70fbf83aae admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:12:27.895347 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-527ec3a6-0d20-409c-b7c7-9e70fbf83aae admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:12:28.207123 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653aee63-437a-4232-88db-d5b12a8a1959 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:12:28.211601 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653aee63-437a-4232-88db-d5b12a8a1959 admin admin] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:12:28.212268 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653aee63-437a-4232-88db-d5b12a8a1959 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:12:28.212372 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-653aee63-437a-4232-88db-d5b12a8a1959 admin admin] PATH_INFO: `/v3/users` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:12:28.213912 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-653aee63-437a-4232-88db-d5b12a8a1959 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:12:28.232815 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-653aee63-437a-4232-88db-d5b12a8a1959 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:12:28.301052 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 849/1700] 10.222.0.52 () {60 vars in 1143 bytes} [Wed Jul 13 22:12:27 2022] POST /identity/v3/users => generated 291 bytes in 423 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 13 22:12:28.305868 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a94dbb2-e785-4880-a2cc-1f209171b07b admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:12:28.321378 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2a94dbb2-e785-4880-a2cc-1f209171b07b admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:12:28.351577 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de84040c-fd8e-440a-af6b-0cd8f48a27c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:12:28.357169 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de84040c-fd8e-440a-af6b-0cd8f48a27c7 admin admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:12:28.360567 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de84040c-fd8e-440a-af6b-0cd8f48a27c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:12:28.361031 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-de84040c-fd8e-440a-af6b-0cd8f48a27c7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:12:28.362167 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de84040c-fd8e-440a-af6b-0cd8f48a27c7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:12:28.366063 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-de84040c-fd8e-440a-af6b-0cd8f48a27c7 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:12:28.411059 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 852/1701] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jul 13 22:12:28 2022] GET /identity/v3/roles => generated 1420 bytes in 107 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 22:12:28.415432 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653aee63-437a-4232-88db-d5b12a8a1959 admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:12:28.426894 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-653aee63-437a-4232-88db-d5b12a8a1959 admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:12:28.444417 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e90925cc-99ef-45cb-9b80-5a01a1b31ff1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:12:28.445119 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90925cc-99ef-45cb-9b80-5a01a1b31ff1 admin admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:12:28.445495 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90925cc-99ef-45cb-9b80-5a01a1b31ff1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:12:28.445678 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e90925cc-99ef-45cb-9b80-5a01a1b31ff1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:12:28.446427 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90925cc-99ef-45cb-9b80-5a01a1b31ff1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:12:28.455678 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e90925cc-99ef-45cb-9b80-5a01a1b31ff1 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:12:28.483872 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 850/1702] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jul 13 22:12:28 2022] GET /identity/v3/roles => generated 1420 bytes in 69 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 22:12:28.489313 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de84040c-fd8e-440a-af6b-0cd8f48a27c7 admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:12:28.531312 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-de84040c-fd8e-440a-af6b-0cd8f48a27c7 admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:12:28.548321 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a834f95e-e2dd-4fd7-bda5-8bc24907b69f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:12:28.549967 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a834f95e-e2dd-4fd7-bda5-8bc24907b69f admin admin] REQUEST_METHOD: `PUT` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:12:28.552241 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a834f95e-e2dd-4fd7-bda5-8bc24907b69f admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:12:28.552521 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a834f95e-e2dd-4fd7-bda5-8bc24907b69f admin admin] PATH_INFO: `/v3/projects/4e3d7c10ff3841019119ef8b83381fe1/users/e7437c0da1864553b85d369b65b768bd/roles/2a5e86976394498abb6310697cdb3f8b` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:12:28.609611 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a834f95e-e2dd-4fd7-bda5-8bc24907b69f admin admin] RBAC: Authorizing `identity:create_grant(project_id=4e3d7c10ff3841019119ef8b83381fe1, user_id=e7437c0da1864553b85d369b65b768bd, role_id=2a5e86976394498abb6310697cdb3f8b)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:12:28.614523 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15077f39-9ba2-45e3-a7cc-5500e85b5b10 None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:12:28.614876 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15077f39-9ba2-45e3-a7cc-5500e85b5b10 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:12:28.615170 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15077f39-9ba2-45e3-a7cc-5500e85b5b10 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:12:28.638387 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a834f95e-e2dd-4fd7-bda5-8bc24907b69f admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:12:28.681893 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 853/1703] 10.222.0.52 () {60 vars in 1482 bytes} [Wed Jul 13 22:12:28 2022] PUT /identity/v3/projects/4e3d7c10ff3841019119ef8b83381fe1/users/e7437c0da1864553b85d369b65b768bd/roles/2a5e86976394498abb6310697cdb3f8b => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 22:12:28.703177 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e317f1d-d0e8-4a5f-a064-aa5b5e04f4c7 None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:12:28.703985 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e317f1d-d0e8-4a5f-a064-aa5b5e04f4c7 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:12:28.704284 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e317f1d-d0e8-4a5f-a064-aa5b5e04f4c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:12:28.739042 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-15077f39-9ba2-45e3-a7cc-5500e85b5b10 None None] MFA Rules not processed for user `45b5d8b0ba324f629f697cf0de5918bb`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 22:12:28.745642 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-15077f39-9ba2-45e3-a7cc-5500e85b5b10 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:12:28.750580 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=92,keystone:DELETE=5,keystone:INSERT=3 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:12:28.821832 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-0e317f1d-d0e8-4a5f-a064-aa5b5e04f4c7 None None] MFA Rules not processed for user `e7437c0da1864553b85d369b65b768bd`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 22:12:28.823652 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-0e317f1d-d0e8-4a5f-a064-aa5b5e04f4c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:12:29.067635 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 854/1704] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jul 13 22:12:28 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 368 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 22:12:29.068931 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e317f1d-d0e8-4a5f-a064-aa5b5e04f4c7 None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:12:29.119602 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 851/1705] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jul 13 22:12:28 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 506 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 22:12:29.142177 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace8031a-3068-4558-a941-76ec6939e3dd None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:12:29.146346 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace8031a-3068-4558-a941-76ec6939e3dd None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:12:29.148990 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ace8031a-3068-4558-a941-76ec6939e3dd None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:12:29.217756 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-ace8031a-3068-4558-a941-76ec6939e3dd None None] MFA Rules not processed for user `45b5d8b0ba324f629f697cf0de5918bb`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 22:12:29.219037 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-ace8031a-3068-4558-a941-76ec6939e3dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:12:29.258195 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e317f1d-d0e8-4a5f-a064-aa5b5e04f4c7 None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:12:29.266669 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 852/1706] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jul 13 22:12:29 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 130 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 22:12:29.281752 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace8031a-3068-4558-a941-76ec6939e3dd None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:12:29.282155 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6fc0455-e299-45a5-92eb-6c2515dcd19c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:12:29.283118 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fc0455-e299-45a5-92eb-6c2515dcd19c admin admin] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:12:29.283613 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fc0455-e299-45a5-92eb-6c2515dcd19c admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:12:29.283969 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-ace8031a-3068-4558-a941-76ec6939e3dd None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:12:29.284314 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6fc0455-e299-45a5-92eb-6c2515dcd19c admin admin] PATH_INFO: `/v3/projects` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:12:29.285227 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6fc0455-e299-45a5-92eb-6c2515dcd19c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:12:29.287602 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6fc0455-e299-45a5-92eb-6c2515dcd19c admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:12:29.363200 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 855/1707] 10.222.0.52 () {60 vars in 1152 bytes} [Wed Jul 13 22:12:29 2022] POST /identity/v3/projects => generated 381 bytes in 295 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 22:12:29.367019 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6fc0455-e299-45a5-92eb-6c2515dcd19c admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:12:29.545237 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:12:29.615748 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ace8031a-3068-4558-a941-76ec6939e3dd None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:12:29.649240 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6fc0455-e299-45a5-92eb-6c2515dcd19c admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:12:29.671433 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b341e7e-c99e-4185-a72a-7dd70a5fee99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:12:29.672463 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b341e7e-c99e-4185-a72a-7dd70a5fee99 admin admin] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:12:29.672962 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b341e7e-c99e-4185-a72a-7dd70a5fee99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:12:29.673454 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b341e7e-c99e-4185-a72a-7dd70a5fee99 admin admin] PATH_INFO: `/v3/users` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:12:29.674375 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b341e7e-c99e-4185-a72a-7dd70a5fee99 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:12:29.681944 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b341e7e-c99e-4185-a72a-7dd70a5fee99 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:12:29.748870 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 856/1708] 10.222.0.52 () {60 vars in 1143 bytes} [Wed Jul 13 22:12:29 2022] POST /identity/v3/users => generated 297 bytes in 382 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 22:12:29.764764 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e174a2d6-77bf-477f-81dc-d71fd355b76d None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:12:29.765478 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e174a2d6-77bf-477f-81dc-d71fd355b76d None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:12:29.768964 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e174a2d6-77bf-477f-81dc-d71fd355b76d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:12:29.822735 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-e174a2d6-77bf-477f-81dc-d71fd355b76d None None] MFA Rules not processed for user `83d2f82cf096424aa4979f443a2729d6`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 22:12:29.824095 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-e174a2d6-77bf-477f-81dc-d71fd355b76d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:12:29.846341 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:12:29.952939 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea437b01-411d-4a79-aedc-3755c55af5dd service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84281091687464990ffbca590b9343b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417e3e2f1333481795a089bef2e24bea', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:12:29.953621 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea437b01-411d-4a79-aedc-3755c55af5dd service nova] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:12:29.953979 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea437b01-411d-4a79-aedc-3755c55af5dd service nova] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:12:29.954172 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ea437b01-411d-4a79-aedc-3755c55af5dd service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:12:29.989708 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 857/1709] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jul 13 22:12:29 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 240 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 22:12:29.992171 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e174a2d6-77bf-477f-81dc-d71fd355b76d None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:12:30.021626 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e174a2d6-77bf-477f-81dc-d71fd355b76d None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:12:30.027093 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3869d8f-ea71-44f6-a17d-dee5249e29e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:12:30.027815 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3869d8f-ea71-44f6-a17d-dee5249e29e8 admin admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:12:30.028265 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3869d8f-ea71-44f6-a17d-dee5249e29e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:12:30.028945 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3869d8f-ea71-44f6-a17d-dee5249e29e8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:12:30.029838 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3869d8f-ea71-44f6-a17d-dee5249e29e8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:12:30.033434 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3869d8f-ea71-44f6-a17d-dee5249e29e8 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:12:30.068569 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 858/1710] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jul 13 22:12:29 2022] GET /identity/v3/roles => generated 1420 bytes in 78 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 22:12:30.073700 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3869d8f-ea71-44f6-a17d-dee5249e29e8 admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:12:30.093015 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3869d8f-ea71-44f6-a17d-dee5249e29e8 admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:12:30.110039 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a90bd81-00bf-4338-90f7-dfc5d0897794 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:12:30.111537 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a90bd81-00bf-4338-90f7-dfc5d0897794 admin admin] REQUEST_METHOD: `PUT` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:12:30.111834 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a90bd81-00bf-4338-90f7-dfc5d0897794 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:12:30.112113 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a90bd81-00bf-4338-90f7-dfc5d0897794 admin admin] PATH_INFO: `/v3/projects/4b382a3b903940068088455f2abbc792/users/06bdd92ae98c4620b3b400486eb55bdf/roles/44efba2579f342df8a0789c59d163ec3` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:12:30.214949 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a90bd81-00bf-4338-90f7-dfc5d0897794 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4b382a3b903940068088455f2abbc792, user_id=06bdd92ae98c4620b3b400486eb55bdf, role_id=44efba2579f342df8a0789c59d163ec3)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:12:30.227559 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a90bd81-00bf-4338-90f7-dfc5d0897794 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:12:30.274011 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea437b01-411d-4a79-aedc-3755c55af5dd service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:12:30.276007 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ea437b01-411d-4a79-aedc-3755c55af5dd service nova] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:12:30.279341 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 859/1711] 10.222.0.52 () {60 vars in 1482 bytes} [Wed Jul 13 22:12:30 2022] PUT /identity/v3/projects/4b382a3b903940068088455f2abbc792/users/06bdd92ae98c4620b3b400486eb55bdf/roles/44efba2579f342df8a0789c59d163ec3 => generated 0 bytes in 207 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 22:12:30.285762 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a90bd81-00bf-4338-90f7-dfc5d0897794 admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:12:30.324645 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 853/1712] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jul 13 22:12:29 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 1043 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 13 22:12:30.532773 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a90bd81-00bf-4338-90f7-dfc5d0897794 admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:12:30.568890 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a32c3615-73c4-46fe-bb3b-1fb39d047360 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:12:30.569974 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32c3615-73c4-46fe-bb3b-1fb39d047360 admin admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:12:30.570308 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32c3615-73c4-46fe-bb3b-1fb39d047360 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:12:30.570669 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a32c3615-73c4-46fe-bb3b-1fb39d047360 admin admin] PATH_INFO: `/v3/roles` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:12:30.571381 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a32c3615-73c4-46fe-bb3b-1fb39d047360 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:12:30.575859 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a32c3615-73c4-46fe-bb3b-1fb39d047360 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:12:30.672192 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea437b01-411d-4a79-aedc-3755c55af5dd service nova] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:12:30.684019 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 860/1713] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jul 13 22:12:30 2022] GET /identity/v3/roles => generated 1420 bytes in 399 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 22:12:30.700551 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ea437b01-411d-4a79-aedc-3755c55af5dd service nova] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:12:30.720956 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4d4b06bd-b209-4994-bef3-d90db65a6115 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:12:30.721894 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4b06bd-b209-4994-bef3-d90db65a6115 admin admin] REQUEST_METHOD: `PUT` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:12:30.722171 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4b06bd-b209-4994-bef3-d90db65a6115 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:12:30.722445 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4d4b06bd-b209-4994-bef3-d90db65a6115 admin admin] PATH_INFO: `/v3/projects/4b382a3b903940068088455f2abbc792/users/06bdd92ae98c4620b3b400486eb55bdf/roles/44efba2579f342df8a0789c59d163ec3` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:12:30.732701 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d4b06bd-b209-4994-bef3-d90db65a6115 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4b382a3b903940068088455f2abbc792, user_id=06bdd92ae98c4620b3b400486eb55bdf, role_id=44efba2579f342df8a0789c59d163ec3)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:12:30.735452 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4d4b06bd-b209-4994-bef3-d90db65a6115 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:12:30.806201 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6ff9f0-6873-4c72-9566-e339f487dc47 None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:12:30.806588 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6ff9f0-6873-4c72-9566-e339f487dc47 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:12:30.806804 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a6ff9f0-6873-4c72-9566-e339f487dc47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:12:30.818105 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 854/1714] 10.222.0.52 () {60 vars in 1482 bytes} [Wed Jul 13 22:12:30 2022] PUT /identity/v3/projects/4b382a3b903940068088455f2abbc792/users/06bdd92ae98c4620b3b400486eb55bdf/roles/44efba2579f342df8a0789c59d163ec3 => generated 0 bytes in 146 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 13 22:12:30.830899 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27a3b3a-963c-4768-908d-78555c49e7ef None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:12:30.831264 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27a3b3a-963c-4768-908d-78555c49e7ef None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:12:30.832259 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f27a3b3a-963c-4768-908d-78555c49e7ef None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:12:30.894321 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-9a6ff9f0-6873-4c72-9566-e339f487dc47 None None] MFA Rules not processed for user `847ec9a1bd4d4dec804318a5c845895a`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 22:12:30.901643 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-9a6ff9f0-6873-4c72-9566-e339f487dc47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:12:30.945211 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-f27a3b3a-963c-4768-908d-78555c49e7ef None None] MFA Rules not processed for user `06bdd92ae98c4620b3b400486eb55bdf`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 22:12:30.947039 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-f27a3b3a-963c-4768-908d-78555c49e7ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:12:31.173758 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 861/1715] 10.222.0.49 () {58 vars in 1015 bytes} [Wed Jul 13 22:12:30 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 369 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 13 22:12:31.216733 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 855/1716] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jul 13 22:12:30 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 387 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 22:12:31.231241 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6ff9f0-6873-4c72-9566-e339f487dc47 None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:12:31.380078 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:12:31.443661 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a6ff9f0-6873-4c72-9566-e339f487dc47 None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:12:31.458811 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6a78ab-2b65-4dc5-9310-9958fde808f4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc77e1ea1ada4700b809beb1c980c5e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417e3e2f1333481795a089bef2e24bea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:12:31.459517 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6a78ab-2b65-4dc5-9310-9958fde808f4 service cinder] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:12:31.459900 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6a78ab-2b65-4dc5-9310-9958fde808f4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:12:31.460275 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb6a78ab-2b65-4dc5-9310-9958fde808f4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:12:31.483317 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6a78ab-2b65-4dc5-9310-9958fde808f4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:12:31.484869 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb6a78ab-2b65-4dc5-9310-9958fde808f4 service cinder] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:12:31.519608 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 862/1717] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jul 13 22:12:31 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 279 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 22:12:31.599000 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27a3b3a-963c-4768-908d-78555c49e7ef None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:12:31.600354 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-f27a3b3a-963c-4768-908d-78555c49e7ef None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:12:31.608888 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:12:31.674950 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6a78ab-2b65-4dc5-9310-9958fde808f4 service cinder] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:12:31.677562 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-bb6a78ab-2b65-4dc5-9310-9958fde808f4 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:12:31.710613 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb6a78ab-2b65-4dc5-9310-9958fde808f4 service cinder] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:12:31.721763 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99380bfc-1859-4f25-a928-d112363d9cf2 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc77e1ea1ada4700b809beb1c980c5e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417e3e2f1333481795a089bef2e24bea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:12:31.722875 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99380bfc-1859-4f25-a928-d112363d9cf2 service cinder] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:12:31.727009 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99380bfc-1859-4f25-a928-d112363d9cf2 service cinder] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:12:31.727425 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99380bfc-1859-4f25-a928-d112363d9cf2 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:12:31.859985 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f27a3b3a-963c-4768-908d-78555c49e7ef None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:12:31.881622 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49f024b3-01e1-4907-bca0-2bcc4c1ba086 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:12:31.884581 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f024b3-01e1-4907-bca0-2bcc4c1ba086 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:12:31.884879 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f024b3-01e1-4907-bca0-2bcc4c1ba086 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:12:31.885160 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49f024b3-01e1-4907-bca0-2bcc4c1ba086 admin admin] PATH_INFO: `/v3/users/340aab38ef56488c91b360983fc3a209` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:12:31.891776 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f024b3-01e1-4907-bca0-2bcc4c1ba086 admin admin] RBAC: Authorizing `identity:delete_user(user_id=340aab38ef56488c91b360983fc3a209)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:12:31.900765 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-49f024b3-01e1-4907-bca0-2bcc4c1ba086 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:12:31.953287 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99380bfc-1859-4f25-a928-d112363d9cf2 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:12:31.955739 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99380bfc-1859-4f25-a928-d112363d9cf2 service cinder] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:12:32.008250 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 863/1718] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jul 13 22:12:31 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 334 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 22:12:32.009495 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99380bfc-1859-4f25-a928-d112363d9cf2 service cinder] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:12:32.014763 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-99380bfc-1859-4f25-a928-d112363d9cf2 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:12:32.059363 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99380bfc-1859-4f25-a928-d112363d9cf2 service cinder] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:12:32.065343 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-49f024b3-01e1-4907-bca0-2bcc4c1ba086 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '340aab38ef56488c91b360983fc3a209'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:12:32.076714 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a6c0d41-8adb-4463-b3c6-35b89c2996e5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '847ec9a1bd4d4dec804318a5c845895a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417e3e2f1333481795a089bef2e24bea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:12:32.080042 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6c0d41-8adb-4463-b3c6-35b89c2996e5 service neutron] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:12:32.080942 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6c0d41-8adb-4463-b3c6-35b89c2996e5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:12:32.083476 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a6c0d41-8adb-4463-b3c6-35b89c2996e5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:12:32.095624 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-49f024b3-01e1-4907-bca0-2bcc4c1ba086 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '340aab38ef56488c91b360983fc3a209'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:12:32.114836 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6c0d41-8adb-4463-b3c6-35b89c2996e5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:12:32.116463 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a6c0d41-8adb-4463-b3c6-35b89c2996e5 service neutron] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:12:32.134271 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-49f024b3-01e1-4907-bca0-2bcc4c1ba086 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '340aab38ef56488c91b360983fc3a209'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:12:32.137178 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-49f024b3-01e1-4907-bca0-2bcc4c1ba086 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '340aab38ef56488c91b360983fc3a209'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:12:32.140123 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-49f024b3-01e1-4907-bca0-2bcc4c1ba086 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '340aab38ef56488c91b360983fc3a209'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:12:32.150630 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 864/1719] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jul 13 22:12:32 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 140 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 22:12:32.155433 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65b4934-1f05-4669-9708-96bcef9562a7 None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:12:32.155710 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65b4934-1f05-4669-9708-96bcef9562a7 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:12:32.155995 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c65b4934-1f05-4669-9708-96bcef9562a7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:12:32.182751 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 856/1720] 10.222.0.52 () {58 vars in 1223 bytes} [Wed Jul 13 22:12:31 2022] DELETE /identity/v3/users/340aab38ef56488c91b360983fc3a209 => generated 0 bytes in 584 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 22:12:32.188455 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058c972a-0b04-4a87-8edc-e5ace96f99e3 None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:12:32.188947 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058c972a-0b04-4a87-8edc-e5ace96f99e3 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:12:32.189303 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-058c972a-0b04-4a87-8edc-e5ace96f99e3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:12:32.243026 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-c65b4934-1f05-4669-9708-96bcef9562a7 None None] MFA Rules not processed for user `064dcac8f59c431cb7d6848f4aaac347`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 22:12:32.246054 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-c65b4934-1f05-4669-9708-96bcef9562a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:12:32.247106 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-058c972a-0b04-4a87-8edc-e5ace96f99e3 None None] MFA Rules not processed for user `cc99c93da5074cf5a48bc2aa9a92c5e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 22:12:32.249467 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-058c972a-0b04-4a87-8edc-e5ace96f99e3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:12:32.494428 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 857/1721] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jul 13 22:12:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 308 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 22:12:32.495996 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d1dde7-1dff-4137-af0a-28574f522e5f None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:12:32.496285 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d1dde7-1dff-4137-af0a-28574f522e5f None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:12:32.496563 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84d1dde7-1dff-4137-af0a-28574f522e5f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:12:32.510570 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 865/1722] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jul 13 22:12:32 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 357 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 22:12:32.530241 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-84d1dde7-1dff-4137-af0a-28574f522e5f None None] MFA Rules not processed for user `18b51e7024fc46a8af3644af27ab886f`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 22:12:32.531536 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-84d1dde7-1dff-4137-af0a-28574f522e5f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:12:32.571976 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65b4934-1f05-4669-9708-96bcef9562a7 None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:12:32.574354 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-c65b4934-1f05-4669-9708-96bcef9562a7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:12:32.608830 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:12:32.694967 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:12:32.749008 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 858/1723] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jul 13 22:12:32 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 254 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 22:12:32.756488 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c65b4934-1f05-4669-9708-96bcef9562a7 None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:12:32.766437 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc390e6-b371-426f-9842-a148749d13e5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '847ec9a1bd4d4dec804318a5c845895a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417e3e2f1333481795a089bef2e24bea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:12:32.783573 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc390e6-b371-426f-9842-a148749d13e5 service neutron] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:12:32.784020 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc390e6-b371-426f-9842-a148749d13e5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:12:32.784888 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbc390e6-b371-426f-9842-a148749d13e5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:12:32.810232 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc390e6-b371-426f-9842-a148749d13e5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:12:32.812142 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbc390e6-b371-426f-9842-a148749d13e5 service neutron] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:12:32.819669 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73773a09-1ee3-4f8c-90c5-371233fd6304 None None] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:12:32.820247 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73773a09-1ee3-4f8c-90c5-371233fd6304 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:12:32.820549 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73773a09-1ee3-4f8c-90c5-371233fd6304 None None] PATH_INFO: `/` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:12:32.823573 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 859/1724] 10.222.0.52 () {54 vars in 912 bytes} [Wed Jul 13 22:12:32 2022] GET /identity => generated 270 bytes in 6 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 22:12:32.830460 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d38e9e-5410-4e4f-b38a-6a62beb3d798 None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:12:32.830858 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d38e9e-5410-4e4f-b38a-6a62beb3d798 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:12:32.831139 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-46d38e9e-5410-4e4f-b38a-6a62beb3d798 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:12:32.847055 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 866/1725] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jul 13 22:12:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 275 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 22:12:32.863114 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-46d38e9e-5410-4e4f-b38a-6a62beb3d798 None None] MFA Rules not processed for user `847ec9a1bd4d4dec804318a5c845895a`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 22:12:32.865278 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-46d38e9e-5410-4e4f-b38a-6a62beb3d798 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:12:32.879661 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 860/1726] 10.222.0.52 () {58 vars in 1009 bytes} [Wed Jul 13 22:12:32 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 51 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 13 22:12:33.073416 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc390e6-b371-426f-9842-a148749d13e5 service neutron] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:12:33.075221 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-bbc390e6-b371-426f-9842-a148749d13e5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:12:33.102271 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbc390e6-b371-426f-9842-a148749d13e5 service neutron] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:12:33.112303 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7213fdac-2323-43ce-8d52-bee599967c3a service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '847ec9a1bd4d4dec804318a5c845895a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417e3e2f1333481795a089bef2e24bea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:12:33.113736 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7213fdac-2323-43ce-8d52-bee599967c3a service neutron] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:12:33.115289 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7213fdac-2323-43ce-8d52-bee599967c3a service neutron] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:12:33.115565 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7213fdac-2323-43ce-8d52-bee599967c3a service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:12:33.146083 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7213fdac-2323-43ce-8d52-bee599967c3a service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:12:33.148595 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d38e9e-5410-4e4f-b38a-6a62beb3d798 None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:12:33.151294 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-46d38e9e-5410-4e4f-b38a-6a62beb3d798 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:12:33.157533 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7213fdac-2323-43ce-8d52-bee599967c3a service neutron] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:12:33.189210 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-46d38e9e-5410-4e4f-b38a-6a62beb3d798 None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:12:33.195248 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 867/1727] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jul 13 22:12:33 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 123 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 22:12:33.208291 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-65ab628e-165f-45fb-91f8-db9236887ba4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:12:33.209226 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ab628e-165f-45fb-91f8-db9236887ba4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:12:33.210993 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ab628e-165f-45fb-91f8-db9236887ba4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:12:33.211443 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-65ab628e-165f-45fb-91f8-db9236887ba4 admin admin] PATH_INFO: `/v3/projects/46d57b6c4d1340649e0de1a8e8ec09f4` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:12:33.217424 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ab628e-165f-45fb-91f8-db9236887ba4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=46d57b6c4d1340649e0de1a8e8ec09f4)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:12:33.225359 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-65ab628e-165f-45fb-91f8-db9236887ba4 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:12:33.360927 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-65ab628e-165f-45fb-91f8-db9236887ba4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '46d57b6c4d1340649e0de1a8e8ec09f4', 'request_id': 'req-65ab628e-165f-45fb-91f8-db9236887ba4'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:12:33.374358 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-65ab628e-165f-45fb-91f8-db9236887ba4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '46d57b6c4d1340649e0de1a8e8ec09f4', 'request_id': 'req-65ab628e-165f-45fb-91f8-db9236887ba4'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:12:33.387871 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-65ab628e-165f-45fb-91f8-db9236887ba4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '46d57b6c4d1340649e0de1a8e8ec09f4', 'request_id': 'req-65ab628e-165f-45fb-91f8-db9236887ba4'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:12:33.424485 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-65ab628e-165f-45fb-91f8-db9236887ba4 admin admin] The token cache is being invalidate because project 46d57b6c4d1340649e0de1a8e8ec09f4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80382) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 22:12:33.426791 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-65ab628e-165f-45fb-91f8-db9236887ba4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:12:33.429635 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-65ab628e-165f-45fb-91f8-db9236887ba4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:12:33.434534 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 861/1728] 10.222.0.52 () {58 vars in 1232 bytes} [Wed Jul 13 22:12:33 2022] DELETE /identity/v3/projects/46d57b6c4d1340649e0de1a8e8ec09f4 => generated 0 bytes in 287 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 13 22:12:33.439652 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a02e081-e5d1-4247-b916-c39fe25f3f76 None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:12:33.440274 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a02e081-e5d1-4247-b916-c39fe25f3f76 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:12:33.440866 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a02e081-e5d1-4247-b916-c39fe25f3f76 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:12:33.502347 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-3a02e081-e5d1-4247-b916-c39fe25f3f76 None None] MFA Rules not processed for user `cc99c93da5074cf5a48bc2aa9a92c5e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 22:12:33.503874 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-3a02e081-e5d1-4247-b916-c39fe25f3f76 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:12:33.859901 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 868/1729] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jul 13 22:12:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 418 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 22:12:33.861828 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30eda7a3-ad11-4c13-866a-6a4241efcd8c None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:12:33.862242 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30eda7a3-ad11-4c13-866a-6a4241efcd8c None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:12:33.862399 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30eda7a3-ad11-4c13-866a-6a4241efcd8c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:12:33.933437 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-30eda7a3-ad11-4c13-866a-6a4241efcd8c None None] MFA Rules not processed for user `cc99c93da5074cf5a48bc2aa9a92c5e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 22:12:33.948174 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-30eda7a3-ad11-4c13-866a-6a4241efcd8c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:12:34.003749 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 862/1730] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jul 13 22:12:33 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 143 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 22:12:34.007626 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a02e081-e5d1-4247-b916-c39fe25f3f76 None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:12:34.032703 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a02e081-e5d1-4247-b916-c39fe25f3f76 None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:12:34.052748 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c432e96-0726-410b-bbf6-ec929248efc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:12:34.053462 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c432e96-0726-410b-bbf6-ec929248efc1 admin admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:12:34.053741 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c432e96-0726-410b-bbf6-ec929248efc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:12:34.055747 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9c432e96-0726-410b-bbf6-ec929248efc1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:12:34.056618 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c432e96-0726-410b-bbf6-ec929248efc1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 22:12:34.056971 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c432e96-0726-410b-bbf6-ec929248efc1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:12:34.062011 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9c432e96-0726-410b-bbf6-ec929248efc1 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:12:34.120278 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 869/1731] 10.222.0.52 () {58 vars in 1152 bytes} [Wed Jul 13 22:12:34 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 113 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 22:12:34.134636 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30eda7a3-ad11-4c13-866a-6a4241efcd8c None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:12:34.158184 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30eda7a3-ad11-4c13-866a-6a4241efcd8c None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:12:34.186838 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32cc4f15-9c95-437f-8be5-31a328b932df admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:12:34.187552 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cc4f15-9c95-437f-8be5-31a328b932df admin admin] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:12:34.187831 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cc4f15-9c95-437f-8be5-31a328b932df admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:12:34.188111 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32cc4f15-9c95-437f-8be5-31a328b932df admin admin] PATH_INFO: `/v3/projects` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:12:34.188848 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32cc4f15-9c95-437f-8be5-31a328b932df admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:12:34.190847 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32cc4f15-9c95-437f-8be5-31a328b932df admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:12:34.358220 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c432e96-0726-410b-bbf6-ec929248efc1 admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:12:34.359649 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 863/1732] 10.222.0.52 () {60 vars in 1152 bytes} [Wed Jul 13 22:12:34 2022] POST /identity/v3/projects => generated 371 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 22:12:34.382305 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9c432e96-0726-410b-bbf6-ec929248efc1 admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:12:34.402568 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32cc4f15-9c95-437f-8be5-31a328b932df admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:12:34.404388 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-32cc4f15-9c95-437f-8be5-31a328b932df admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:12:34.840354 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32cc4f15-9c95-437f-8be5-31a328b932df admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:12:34.843950 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7ea4d8-15ff-4def-9df9-dfaa2073c802 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:12:34.844322 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7ea4d8-15ff-4def-9df9-dfaa2073c802 admin admin] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:12:34.849584 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7ea4d8-15ff-4def-9df9-dfaa2073c802 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:12:34.849903 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b7ea4d8-15ff-4def-9df9-dfaa2073c802 admin admin] PATH_INFO: `/v3/users` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:12:34.850644 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b7ea4d8-15ff-4def-9df9-dfaa2073c802 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:12:34.853613 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b7ea4d8-15ff-4def-9df9-dfaa2073c802 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:12:34.861643 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87fddd63-643b-43e2-ba74-71e260a40881 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:12:34.864536 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fddd63-643b-43e2-ba74-71e260a40881 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:12:34.865781 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fddd63-643b-43e2-ba74-71e260a40881 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:12:34.868685 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-87fddd63-643b-43e2-ba74-71e260a40881 admin admin] PATH_INFO: `/v3/projects/1396528f2e424a5caa45211fed5f48e2` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:12:34.880940 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87fddd63-643b-43e2-ba74-71e260a40881 admin admin] RBAC: Authorizing `identity:delete_project(project_id=1396528f2e424a5caa45211fed5f48e2)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:12:34.883126 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-87fddd63-643b-43e2-ba74-71e260a40881 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:12:34.961916 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 870/1733] 10.222.0.52 () {60 vars in 1143 bytes} [Wed Jul 13 22:12:34 2022] POST /identity/v3/users => generated 286 bytes in 603 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 22:12:34.964938 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7ea4d8-15ff-4def-9df9-dfaa2073c802 admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:12:34.983094 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-5b7ea4d8-15ff-4def-9df9-dfaa2073c802 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:12:35.029211 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:12:35.107229 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-87fddd63-643b-43e2-ba74-71e260a40881 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '1396528f2e424a5caa45211fed5f48e2', 'request_id': 'req-87fddd63-643b-43e2-ba74-71e260a40881'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:12:35.140498 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-87fddd63-643b-43e2-ba74-71e260a40881 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '1396528f2e424a5caa45211fed5f48e2', 'request_id': 'req-87fddd63-643b-43e2-ba74-71e260a40881'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:12:35.202178 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-87fddd63-643b-43e2-ba74-71e260a40881 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '1396528f2e424a5caa45211fed5f48e2', 'request_id': 'req-87fddd63-643b-43e2-ba74-71e260a40881'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:12:35.274114 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-87fddd63-643b-43e2-ba74-71e260a40881 admin admin] The token cache is being invalidate because project 1396528f2e424a5caa45211fed5f48e2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80382) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 22:12:35.279979 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-87fddd63-643b-43e2-ba74-71e260a40881 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:12:35.283037 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-87fddd63-643b-43e2-ba74-71e260a40881 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:12:35.285260 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:12:35.292616 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 864/1734] 10.222.0.52 () {58 vars in 1232 bytes} [Wed Jul 13 22:12:34 2022] DELETE /identity/v3/projects/1396528f2e424a5caa45211fed5f48e2 => generated 0 bytes in 890 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 13 22:12:35.293627 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87fddd63-643b-43e2-ba74-71e260a40881 admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:12:35.314049 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-87fddd63-643b-43e2-ba74-71e260a40881 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:12:35.372362 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b7ea4d8-15ff-4def-9df9-dfaa2073c802 admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:12:35.497076 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-87fddd63-643b-43e2-ba74-71e260a40881 admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:12:35.503959 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a43aac1-bfed-4f91-aac1-480d8cc7777f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:12:35.509962 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a43aac1-bfed-4f91-aac1-480d8cc7777f admin admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:12:35.510248 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a43aac1-bfed-4f91-aac1-480d8cc7777f admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:12:35.510546 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5a43aac1-bfed-4f91-aac1-480d8cc7777f admin admin] PATH_INFO: `/v3/roles` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:12:35.511400 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a43aac1-bfed-4f91-aac1-480d8cc7777f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:12:35.516024 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5a43aac1-bfed-4f91-aac1-480d8cc7777f admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:12:35.561262 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 865/1735] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jul 13 22:12:35 2022] GET /identity/v3/roles => generated 1420 bytes in 267 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 22:12:35.562801 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a43aac1-bfed-4f91-aac1-480d8cc7777f admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:12:35.569877 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-5a43aac1-bfed-4f91-aac1-480d8cc7777f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:12:35.618420 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5a43aac1-bfed-4f91-aac1-480d8cc7777f admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:12:35.637149 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffd42a4-7f52-425d-a5c4-6639a3909632 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:12:35.638972 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffd42a4-7f52-425d-a5c4-6639a3909632 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:12:35.642255 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffd42a4-7f52-425d-a5c4-6639a3909632 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:12:35.643272 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffd42a4-7f52-425d-a5c4-6639a3909632 admin admin] PATH_INFO: `/v3/users/8fc94ac51e584ef488d695cccd8a5690` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:12:35.644914 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bffd42a4-7f52-425d-a5c4-6639a3909632 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8fc94ac51e584ef488d695cccd8a5690)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:12:35.658790 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bffd42a4-7f52-425d-a5c4-6639a3909632 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:12:35.677430 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-559f6e81-23b2-47f1-b645-8970af0297db service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84281091687464990ffbca590b9343b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417e3e2f1333481795a089bef2e24bea', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:12:35.681339 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559f6e81-23b2-47f1-b645-8970af0297db service nova] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:12:35.681619 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559f6e81-23b2-47f1-b645-8970af0297db service nova] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:12:35.684143 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-559f6e81-23b2-47f1-b645-8970af0297db service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:12:35.691537 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-559f6e81-23b2-47f1-b645-8970af0297db service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:12:35.866391 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-bffd42a4-7f52-425d-a5c4-6639a3909632 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8fc94ac51e584ef488d695cccd8a5690'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:12:35.891945 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-bffd42a4-7f52-425d-a5c4-6639a3909632 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8fc94ac51e584ef488d695cccd8a5690'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:12:35.940543 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-bffd42a4-7f52-425d-a5c4-6639a3909632 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8fc94ac51e584ef488d695cccd8a5690'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:12:35.942738 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-bffd42a4-7f52-425d-a5c4-6639a3909632 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8fc94ac51e584ef488d695cccd8a5690'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:12:35.945636 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-bffd42a4-7f52-425d-a5c4-6639a3909632 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8fc94ac51e584ef488d695cccd8a5690'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:12:35.993290 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-559f6e81-23b2-47f1-b645-8970af0297db service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:12:35.995840 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-559f6e81-23b2-47f1-b645-8970af0297db service nova] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:12:36.011668 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 871/1736] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jul 13 22:12:34 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 1052 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 22:12:36.025055 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4695c3ed-d9c2-46ea-a9e2-afe995630212 None None] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:12:36.025055 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4695c3ed-d9c2-46ea-a9e2-afe995630212 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:12:36.025055 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4695c3ed-d9c2-46ea-a9e2-afe995630212 None None] PATH_INFO: `/` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:12:36.025055 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 872/1737] 10.222.0.52 () {54 vars in 921 bytes} [Wed Jul 13 22:12:36 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 22:12:36.025055 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4695c3ed-d9c2-46ea-a9e2-afe995630212 None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:12:36.025055 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-4695c3ed-d9c2-46ea-a9e2-afe995630212 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:12:36.029139 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 866/1738] 10.222.0.52 () {58 vars in 1223 bytes} [Wed Jul 13 22:12:35 2022] DELETE /identity/v3/users/8fc94ac51e584ef488d695cccd8a5690 => generated 0 bytes in 467 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 22:12:36.063741 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4695c3ed-d9c2-46ea-a9e2-afe995630212 None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:12:36.136609 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654fb855-4df0-4420-af93-d20f0c2197a9 None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:12:36.136933 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654fb855-4df0-4420-af93-d20f0c2197a9 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:12:36.137189 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-654fb855-4df0-4420-af93-d20f0c2197a9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:12:36.191692 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-654fb855-4df0-4420-af93-d20f0c2197a9 None None] MFA Rules not processed for user `f84281091687464990ffbca590b9343b`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 22:12:36.197194 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-654fb855-4df0-4420-af93-d20f0c2197a9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:12:36.248186 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=81,keystone:DELETE=17,keystone:INSERT=2 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:12:36.349014 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253a7409-03cf-4734-810a-bffc931288e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:12:36.352529 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253a7409-03cf-4734-810a-bffc931288e6 admin admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:12:36.358759 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253a7409-03cf-4734-810a-bffc931288e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:12:36.360098 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-253a7409-03cf-4734-810a-bffc931288e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:12:36.361010 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253a7409-03cf-4734-810a-bffc931288e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:12:36.364634 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-253a7409-03cf-4734-810a-bffc931288e6 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:12:36.440016 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 873/1739] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jul 13 22:12:36 2022] GET /identity/v3/roles => generated 1420 bytes in 420 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 22:12:36.443096 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253a7409-03cf-4734-810a-bffc931288e6 admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:12:36.478290 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 867/1740] 10.222.0.52 () {58 vars in 1018 bytes} [Wed Jul 13 22:12:36 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 343 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 22:12:36.488839 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-253a7409-03cf-4734-810a-bffc931288e6 admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:12:36.505793 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7acd86b6-8b57-4fe5-8c22-18dbb4e915fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:12:36.508857 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acd86b6-8b57-4fe5-8c22-18dbb4e915fc admin admin] REQUEST_METHOD: `PUT` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:12:36.510285 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acd86b6-8b57-4fe5-8c22-18dbb4e915fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:12:36.510639 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7acd86b6-8b57-4fe5-8c22-18dbb4e915fc admin admin] PATH_INFO: `/v3/projects/ec73f12d9bed4198833dd9a0dad198bb/users/c0fe2106d5f74363b1d5550f7f523ff9/roles/2a5e86976394498abb6310697cdb3f8b` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:12:36.559332 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7acd86b6-8b57-4fe5-8c22-18dbb4e915fc admin admin] RBAC: Authorizing `identity:create_grant(project_id=ec73f12d9bed4198833dd9a0dad198bb, user_id=c0fe2106d5f74363b1d5550f7f523ff9, role_id=2a5e86976394498abb6310697cdb3f8b)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:12:36.572250 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7acd86b6-8b57-4fe5-8c22-18dbb4e915fc admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:12:36.600859 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:12:36.619428 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 874/1741] 10.222.0.52 () {60 vars in 1482 bytes} [Wed Jul 13 22:12:36 2022] PUT /identity/v3/projects/ec73f12d9bed4198833dd9a0dad198bb/users/c0fe2106d5f74363b1d5550f7f523ff9/roles/2a5e86976394498abb6310697cdb3f8b => generated 0 bytes in 177 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 22:12:36.631907 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc074d1e-cade-4ca6-87fc-7a20ffa12a4f None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:12:36.632994 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc074d1e-cade-4ca6-87fc-7a20ffa12a4f None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:12:36.633821 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc074d1e-cade-4ca6-87fc-7a20ffa12a4f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:12:36.736668 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-dc074d1e-cade-4ca6-87fc-7a20ffa12a4f None None] MFA Rules not processed for user `c0fe2106d5f74363b1d5550f7f523ff9`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 22:12:36.740321 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-dc074d1e-cade-4ca6-87fc-7a20ffa12a4f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:12:36.886230 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 868/1742] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jul 13 22:12:36 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 255 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 22:12:36.890852 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7acd86b6-8b57-4fe5-8c22-18dbb4e915fc admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:12:37.041588 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7acd86b6-8b57-4fe5-8c22-18dbb4e915fc admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:12:37.049386 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e255974f-369f-45d7-9d07-e22a4bcc1375 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:12:37.050139 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e255974f-369f-45d7-9d07-e22a4bcc1375 admin admin] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:12:37.050511 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e255974f-369f-45d7-9d07-e22a4bcc1375 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:12:37.050943 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e255974f-369f-45d7-9d07-e22a4bcc1375 admin admin] PATH_INFO: `/v3/projects` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:12:37.052599 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e255974f-369f-45d7-9d07-e22a4bcc1375 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:12:37.055372 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e255974f-369f-45d7-9d07-e22a4bcc1375 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:12:37.102331 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 875/1743] 10.222.0.52 () {60 vars in 1152 bytes} [Wed Jul 13 22:12:36 2022] POST /identity/v3/projects => generated 371 bytes in 209 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 22:12:37.104053 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc074d1e-cade-4ca6-87fc-7a20ffa12a4f None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:12:37.126011 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc074d1e-cade-4ca6-87fc-7a20ffa12a4f None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:12:37.169509 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c479ab56-6efe-466d-b7bc-916bb140f688 None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:12:37.169811 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c479ab56-6efe-466d-b7bc-916bb140f688 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:12:37.170626 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c479ab56-6efe-466d-b7bc-916bb140f688 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:12:37.192762 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-c479ab56-6efe-466d-b7bc-916bb140f688 None None] MFA Rules not processed for user `45b5d8b0ba324f629f697cf0de5918bb`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 22:12:37.195023 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-c479ab56-6efe-466d-b7bc-916bb140f688 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:12:37.299615 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7656646a-e9b6-42f3-ace5-76e463f026dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:12:37.300420 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7656646a-e9b6-42f3-ace5-76e463f026dc admin admin] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:12:37.300822 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7656646a-e9b6-42f3-ace5-76e463f026dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:12:37.301263 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7656646a-e9b6-42f3-ace5-76e463f026dc admin admin] PATH_INFO: `/v3/users` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:12:37.302360 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7656646a-e9b6-42f3-ace5-76e463f026dc admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:12:37.304525 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7656646a-e9b6-42f3-ace5-76e463f026dc admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:12:37.367965 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 869/1744] 10.222.0.52 () {60 vars in 1143 bytes} [Wed Jul 13 22:12:37 2022] POST /identity/v3/users => generated 292 bytes in 264 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 22:12:37.372019 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7656646a-e9b6-42f3-ace5-76e463f026dc admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:12:37.376941 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 876/1745] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jul 13 22:12:37 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 208 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 22:12:37.390298 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7656646a-e9b6-42f3-ace5-76e463f026dc admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:12:37.398094 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df4fcb0-54d5-48cc-812d-870fe79fe0b6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:12:37.399287 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df4fcb0-54d5-48cc-812d-870fe79fe0b6 admin admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:12:37.399828 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df4fcb0-54d5-48cc-812d-870fe79fe0b6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:12:37.400273 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3df4fcb0-54d5-48cc-812d-870fe79fe0b6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:12:37.401487 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df4fcb0-54d5-48cc-812d-870fe79fe0b6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:12:37.403986 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3df4fcb0-54d5-48cc-812d-870fe79fe0b6 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:12:37.420357 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 870/1746] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jul 13 22:12:37 2022] GET /identity/v3/roles => generated 1420 bytes in 52 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 22:12:37.424327 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c479ab56-6efe-466d-b7bc-916bb140f688 None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:12:37.449419 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c479ab56-6efe-466d-b7bc-916bb140f688 None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:12:37.471023 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3dabb4-9b7a-4344-9aa3-24bfca32d99d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:12:37.474519 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3dabb4-9b7a-4344-9aa3-24bfca32d99d admin admin] REQUEST_METHOD: `PUT` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:12:37.478547 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3dabb4-9b7a-4344-9aa3-24bfca32d99d admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:12:37.480036 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff3dabb4-9b7a-4344-9aa3-24bfca32d99d admin admin] PATH_INFO: `/v3/projects/6f807bd2a3c748e78c50426bf90ff6d9/users/001a0ff2f2844b3c85651a4e1d3dfc46/roles/44efba2579f342df8a0789c59d163ec3` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:12:37.513510 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3dabb4-9b7a-4344-9aa3-24bfca32d99d admin admin] RBAC: Authorizing `identity:create_grant(project_id=6f807bd2a3c748e78c50426bf90ff6d9, user_id=001a0ff2f2844b3c85651a4e1d3dfc46, role_id=44efba2579f342df8a0789c59d163ec3)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:12:37.517284 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff3dabb4-9b7a-4344-9aa3-24bfca32d99d admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:12:37.578584 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 877/1747] 10.222.0.52 () {60 vars in 1482 bytes} [Wed Jul 13 22:12:37 2022] PUT /identity/v3/projects/6f807bd2a3c748e78c50426bf90ff6d9/users/001a0ff2f2844b3c85651a4e1d3dfc46/roles/44efba2579f342df8a0789c59d163ec3 => generated 0 bytes in 152 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 22:12:37.580090 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df4fcb0-54d5-48cc-812d-870fe79fe0b6 admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:12:37.599086 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3df4fcb0-54d5-48cc-812d-870fe79fe0b6 admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:12:37.624243 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:12:37.708736 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3dabb4-9b7a-4344-9aa3-24bfca32d99d admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:12:37.710561 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-ff3dabb4-9b7a-4344-9aa3-24bfca32d99d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:12:37.735713 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4b9ec2-1b4d-41db-9cd3-901a18fda2cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:12:37.736477 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4b9ec2-1b4d-41db-9cd3-901a18fda2cf admin admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:12:37.736961 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4b9ec2-1b4d-41db-9cd3-901a18fda2cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:12:37.737244 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6e4b9ec2-1b4d-41db-9cd3-901a18fda2cf admin admin] PATH_INFO: `/v3/roles` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:12:37.738327 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4b9ec2-1b4d-41db-9cd3-901a18fda2cf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:12:37.741424 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6e4b9ec2-1b4d-41db-9cd3-901a18fda2cf admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:12:37.754897 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 871/1748] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jul 13 22:12:37 2022] GET /identity/v3/roles => generated 1420 bytes in 176 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 22:12:37.759057 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4b9ec2-1b4d-41db-9cd3-901a18fda2cf admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:12:37.774727 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6e4b9ec2-1b4d-41db-9cd3-901a18fda2cf admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:12:37.791758 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b95d946-39cd-4838-9f6f-d01202ca64fd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:12:37.792744 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b95d946-39cd-4838-9f6f-d01202ca64fd admin admin] REQUEST_METHOD: `PUT` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:12:37.793027 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b95d946-39cd-4838-9f6f-d01202ca64fd admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:12:37.793365 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b95d946-39cd-4838-9f6f-d01202ca64fd admin admin] PATH_INFO: `/v3/projects/6f807bd2a3c748e78c50426bf90ff6d9/users/001a0ff2f2844b3c85651a4e1d3dfc46/roles/44efba2579f342df8a0789c59d163ec3` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:12:37.795737 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b95d946-39cd-4838-9f6f-d01202ca64fd admin admin] RBAC: Authorizing `identity:create_grant(project_id=6f807bd2a3c748e78c50426bf90ff6d9, user_id=001a0ff2f2844b3c85651a4e1d3dfc46, role_id=44efba2579f342df8a0789c59d163ec3)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:12:37.798657 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b95d946-39cd-4838-9f6f-d01202ca64fd admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:12:37.823142 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 872/1749] 10.222.0.52 () {60 vars in 1482 bytes} [Wed Jul 13 22:12:37 2022] PUT /identity/v3/projects/6f807bd2a3c748e78c50426bf90ff6d9/users/001a0ff2f2844b3c85651a4e1d3dfc46/roles/44efba2579f342df8a0789c59d163ec3 => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 22:12:37.833420 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ddb85c-2f67-4614-82c7-775a7b40899b None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:12:37.833731 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ddb85c-2f67-4614-82c7-775a7b40899b None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:12:37.834023 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d4ddb85c-2f67-4614-82c7-775a7b40899b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:12:37.837753 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:12:37.845092 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff3dabb4-9b7a-4344-9aa3-24bfca32d99d admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:12:37.879726 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-d4ddb85c-2f67-4614-82c7-775a7b40899b None None] MFA Rules not processed for user `001a0ff2f2844b3c85651a4e1d3dfc46`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 22:12:37.882289 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-d4ddb85c-2f67-4614-82c7-775a7b40899b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:12:37.962471 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36a3dca-7628-415f-8200-b1068e5da391 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84281091687464990ffbca590b9343b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417e3e2f1333481795a089bef2e24bea', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:12:37.963170 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36a3dca-7628-415f-8200-b1068e5da391 service nova] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:12:37.963690 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36a3dca-7628-415f-8200-b1068e5da391 service nova] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:12:37.963690 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d36a3dca-7628-415f-8200-b1068e5da391 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:12:37.976709 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d36a3dca-7628-415f-8200-b1068e5da391 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:12:37.979934 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d36a3dca-7628-415f-8200-b1068e5da391 service nova] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:12:37.997689 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 878/1750] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jul 13 22:12:37 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 289 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 13 22:12:38.027510 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 873/1751] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jul 13 22:12:37 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 195 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 13 22:12:38.043509 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36a3dca-7628-415f-8200-b1068e5da391 service nova] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:12:38.046385 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-d36a3dca-7628-415f-8200-b1068e5da391 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:12:38.133091 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ddb85c-2f67-4614-82c7-775a7b40899b None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:12:38.135231 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-d4ddb85c-2f67-4614-82c7-775a7b40899b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:12:38.200121 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d36a3dca-7628-415f-8200-b1068e5da391 service nova] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:12:38.204682 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ff7922-c5ec-45ce-bc1b-7d1e46e79364 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc77e1ea1ada4700b809beb1c980c5e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417e3e2f1333481795a089bef2e24bea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:12:38.205367 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ff7922-c5ec-45ce-bc1b-7d1e46e79364 service cinder] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:12:38.205649 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ff7922-c5ec-45ce-bc1b-7d1e46e79364 service cinder] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:12:38.206267 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27ff7922-c5ec-45ce-bc1b-7d1e46e79364 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:12:38.218125 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ff7922-c5ec-45ce-bc1b-7d1e46e79364 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:12:38.219643 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27ff7922-c5ec-45ce-bc1b-7d1e46e79364 service cinder] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:12:38.248250 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 879/1752] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jul 13 22:12:38 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 206 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 22:12:38.366884 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d4ddb85c-2f67-4614-82c7-775a7b40899b None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:12:38.376639 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-baa392d5-166a-447e-ac15-1d46a0c27d19 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:12:38.379472 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa392d5-166a-447e-ac15-1d46a0c27d19 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:12:38.380100 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa392d5-166a-447e-ac15-1d46a0c27d19 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:12:38.380575 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-baa392d5-166a-447e-ac15-1d46a0c27d19 admin admin] PATH_INFO: `/v3/projects/3ada8d5f66144266b8218959ebf1432b` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:12:38.382973 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa392d5-166a-447e-ac15-1d46a0c27d19 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3ada8d5f66144266b8218959ebf1432b)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:12:38.385666 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-baa392d5-166a-447e-ac15-1d46a0c27d19 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:12:38.442207 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=98,keystone:INSERT=1,keystone:DELETE=1 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:12:38.451698 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ff7922-c5ec-45ce-bc1b-7d1e46e79364 service cinder] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:12:38.464402 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27ff7922-c5ec-45ce-bc1b-7d1e46e79364 service cinder] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:12:38.481790 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-baa392d5-166a-447e-ac15-1d46a0c27d19 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3ada8d5f66144266b8218959ebf1432b', 'request_id': 'req-baa392d5-166a-447e-ac15-1d46a0c27d19'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:12:38.495231 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-baa392d5-166a-447e-ac15-1d46a0c27d19 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3ada8d5f66144266b8218959ebf1432b', 'request_id': 'req-baa392d5-166a-447e-ac15-1d46a0c27d19'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:12:38.505080 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-baa392d5-166a-447e-ac15-1d46a0c27d19 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3ada8d5f66144266b8218959ebf1432b', 'request_id': 'req-baa392d5-166a-447e-ac15-1d46a0c27d19'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:12:38.530766 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-baa392d5-166a-447e-ac15-1d46a0c27d19 admin admin] The token cache is being invalidate because project 3ada8d5f66144266b8218959ebf1432b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80382) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 22:12:38.531048 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-baa392d5-166a-447e-ac15-1d46a0c27d19 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:12:38.531787 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-baa392d5-166a-447e-ac15-1d46a0c27d19 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:12:38.537169 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 874/1753] 10.222.0.52 () {58 vars in 1232 bytes} [Wed Jul 13 22:12:38 2022] DELETE /identity/v3/projects/3ada8d5f66144266b8218959ebf1432b => generated 0 bytes in 402 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 22:12:38.539469 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7152fea-2cfd-4bcf-a90b-3c119954fc44 None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:12:38.540015 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7152fea-2cfd-4bcf-a90b-3c119954fc44 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:12:38.540408 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7152fea-2cfd-4bcf-a90b-3c119954fc44 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:12:38.577438 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-a7152fea-2cfd-4bcf-a90b-3c119954fc44 None None] MFA Rules not processed for user `cc99c93da5074cf5a48bc2aa9a92c5e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 22:12:38.584857 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-a7152fea-2cfd-4bcf-a90b-3c119954fc44 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:12:38.597723 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2096260-e4f3-470e-a615-772b63ce5fe5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc77e1ea1ada4700b809beb1c980c5e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417e3e2f1333481795a089bef2e24bea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:12:38.598203 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2096260-e4f3-470e-a615-772b63ce5fe5 service cinder] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:12:38.598710 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2096260-e4f3-470e-a615-772b63ce5fe5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:12:38.598992 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2096260-e4f3-470e-a615-772b63ce5fe5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:12:38.602943 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-a2096260-e4f3-470e-a615-772b63ce5fe5 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:12:38.689156 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:12:38.700569 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 875/1754] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jul 13 22:12:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 161 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 22:12:38.709002 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e1dcfe-0e84-4be1-801c-5224d6a5dc07 None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:12:38.709481 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e1dcfe-0e84-4be1-801c-5224d6a5dc07 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:12:38.709947 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82e1dcfe-0e84-4be1-801c-5224d6a5dc07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:12:38.711490 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2096260-e4f3-470e-a615-772b63ce5fe5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:12:38.713168 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2096260-e4f3-470e-a615-772b63ce5fe5 service cinder] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:12:38.734683 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-82e1dcfe-0e84-4be1-801c-5224d6a5dc07 None None] MFA Rules not processed for user `cc99c93da5074cf5a48bc2aa9a92c5e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 22:12:38.736046 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-82e1dcfe-0e84-4be1-801c-5224d6a5dc07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:12:38.745384 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 880/1755] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jul 13 22:12:38 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 294 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 13 22:12:38.746413 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 876/1756] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jul 13 22:12:38 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 22:12:38.750352 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2096260-e4f3-470e-a615-772b63ce5fe5 service cinder] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:12:38.765842 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2096260-e4f3-470e-a615-772b63ce5fe5 service cinder] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:12:38.773439 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbdb4dc-0b08-4394-b5c8-983792994384 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:12:38.775614 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbdb4dc-0b08-4394-b5c8-983792994384 admin admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:12:38.776184 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbdb4dc-0b08-4394-b5c8-983792994384 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:12:38.776665 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbbdb4dc-0b08-4394-b5c8-983792994384 admin admin] PATH_INFO: `/v3/domains` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:12:38.777728 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbbdb4dc-0b08-4394-b5c8-983792994384 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 22:12:38.778884 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbbdb4dc-0b08-4394-b5c8-983792994384 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:12:38.781533 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbbdb4dc-0b08-4394-b5c8-983792994384 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:12:38.799512 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 881/1757] 10.222.0.52 () {58 vars in 1152 bytes} [Wed Jul 13 22:12:38 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 49 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 22:12:38.803665 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e1dcfe-0e84-4be1-801c-5224d6a5dc07 None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:12:38.813983 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82e1dcfe-0e84-4be1-801c-5224d6a5dc07 None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:12:38.819476 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81c37fb-a2d1-49cb-bc9f-832159dc8a4b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:12:38.820176 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81c37fb-a2d1-49cb-bc9f-832159dc8a4b admin admin] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:12:38.820463 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81c37fb-a2d1-49cb-bc9f-832159dc8a4b admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:12:38.820780 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d81c37fb-a2d1-49cb-bc9f-832159dc8a4b admin admin] PATH_INFO: `/v3/projects` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:12:38.822889 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d81c37fb-a2d1-49cb-bc9f-832159dc8a4b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:12:38.825054 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d81c37fb-a2d1-49cb-bc9f-832159dc8a4b admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:12:38.865564 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 877/1758] 10.222.0.52 () {60 vars in 1152 bytes} [Wed Jul 13 22:12:38 2022] POST /identity/v3/projects => generated 409 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 22:12:38.869592 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbdb4dc-0b08-4394-b5c8-983792994384 admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:12:38.884319 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbbdb4dc-0b08-4394-b5c8-983792994384 admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:12:39.024747 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0b4283-2462-49dd-9dba-2f8a17385fac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:12:39.025808 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0b4283-2462-49dd-9dba-2f8a17385fac admin admin] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:12:39.026093 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0b4283-2462-49dd-9dba-2f8a17385fac admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:12:39.026367 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c0b4283-2462-49dd-9dba-2f8a17385fac admin admin] PATH_INFO: `/v3/users` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:12:39.029206 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0b4283-2462-49dd-9dba-2f8a17385fac admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:12:39.034720 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c0b4283-2462-49dd-9dba-2f8a17385fac admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:12:39.095829 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 882/1759] 10.222.0.52 () {60 vars in 1143 bytes} [Wed Jul 13 22:12:38 2022] POST /identity/v3/users => generated 305 bytes in 224 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 22:12:39.098565 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81c37fb-a2d1-49cb-bc9f-832159dc8a4b admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:12:39.114185 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d81c37fb-a2d1-49cb-bc9f-832159dc8a4b admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:12:39.120880 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17138963-063b-4663-abc4-147da17fc376 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:12:39.121921 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17138963-063b-4663-abc4-147da17fc376 admin admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:12:39.123724 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17138963-063b-4663-abc4-147da17fc376 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:12:39.124241 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17138963-063b-4663-abc4-147da17fc376 admin admin] PATH_INFO: `/v3/roles` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:12:39.125280 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17138963-063b-4663-abc4-147da17fc376 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:12:39.127311 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17138963-063b-4663-abc4-147da17fc376 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:12:39.148139 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 878/1760] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jul 13 22:12:39 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 22:12:39.148968 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0b4283-2462-49dd-9dba-2f8a17385fac admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:12:39.164346 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c0b4283-2462-49dd-9dba-2f8a17385fac admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:12:39.174242 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4db9ae9-c3cc-480a-b266-54b0a0569b0a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:12:39.174992 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4db9ae9-c3cc-480a-b266-54b0a0569b0a admin admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:12:39.175276 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4db9ae9-c3cc-480a-b266-54b0a0569b0a admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:12:39.175736 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4db9ae9-c3cc-480a-b266-54b0a0569b0a admin admin] PATH_INFO: `/v3/roles` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:12:39.176715 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4db9ae9-c3cc-480a-b266-54b0a0569b0a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:12:39.178374 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4db9ae9-c3cc-480a-b266-54b0a0569b0a admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:12:39.193108 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 883/1761] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jul 13 22:12:39 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 22:12:39.195356 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17138963-063b-4663-abc4-147da17fc376 admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:12:39.208551 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17138963-063b-4663-abc4-147da17fc376 admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:12:39.213893 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6013ad60-b89e-432e-a402-8f4b52c1997d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:12:39.215424 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6013ad60-b89e-432e-a402-8f4b52c1997d admin admin] REQUEST_METHOD: `PUT` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:12:39.215866 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6013ad60-b89e-432e-a402-8f4b52c1997d admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:12:39.216243 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6013ad60-b89e-432e-a402-8f4b52c1997d admin admin] PATH_INFO: `/v3/projects/265a530513fd48bdaa63754a18cfb047/users/e709cc7e6bdc48f2842d69889e2b0c1f/roles/2a5e86976394498abb6310697cdb3f8b` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:12:39.234756 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6013ad60-b89e-432e-a402-8f4b52c1997d admin admin] RBAC: Authorizing `identity:create_grant(project_id=265a530513fd48bdaa63754a18cfb047, user_id=e709cc7e6bdc48f2842d69889e2b0c1f, role_id=2a5e86976394498abb6310697cdb3f8b)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:12:39.238308 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6013ad60-b89e-432e-a402-8f4b52c1997d admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:12:39.271483 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 879/1762] 10.222.0.52 () {60 vars in 1482 bytes} [Wed Jul 13 22:12:39 2022] PUT /identity/v3/projects/265a530513fd48bdaa63754a18cfb047/users/e709cc7e6bdc48f2842d69889e2b0c1f/roles/2a5e86976394498abb6310697cdb3f8b => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 22:12:39.273210 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acd4e4d-1014-47a5-982d-bb9e86d2f784 None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:12:39.273570 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acd4e4d-1014-47a5-982d-bb9e86d2f784 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:12:39.273942 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1acd4e4d-1014-47a5-982d-bb9e86d2f784 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:12:39.314740 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-1acd4e4d-1014-47a5-982d-bb9e86d2f784 None None] MFA Rules not processed for user `e709cc7e6bdc48f2842d69889e2b0c1f`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 22:12:39.316183 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-1acd4e4d-1014-47a5-982d-bb9e86d2f784 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:12:39.423703 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 884/1763] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jul 13 22:12:39 2022] POST /identity/v3/auth/tokens => generated 3276 bytes in 152 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 13 22:12:39.428380 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6013ad60-b89e-432e-a402-8f4b52c1997d admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:12:39.501692 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:DELETE=6,keystone:SELECT=91,keystone:INSERT=3 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:12:39.525282 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6013ad60-b89e-432e-a402-8f4b52c1997d admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:12:39.544515 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8aa546-e850-417b-bd05-1884201844af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:12:39.545731 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8aa546-e850-417b-bd05-1884201844af admin admin] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:12:39.546052 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8aa546-e850-417b-bd05-1884201844af admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:12:39.546389 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8aa546-e850-417b-bd05-1884201844af admin admin] PATH_INFO: `/v3/projects` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:12:39.547195 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff8aa546-e850-417b-bd05-1884201844af admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:12:39.549664 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff8aa546-e850-417b-bd05-1884201844af admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:12:39.613767 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 880/1764] 10.222.0.52 () {60 vars in 1152 bytes} [Wed Jul 13 22:12:39 2022] POST /identity/v3/projects => generated 409 bytes in 182 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 22:12:39.619777 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1acd4e4d-1014-47a5-982d-bb9e86d2f784 None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:12:39.636757 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1acd4e4d-1014-47a5-982d-bb9e86d2f784 None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:12:39.798894 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:12:39.863150 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967888b0-5de9-482e-948a-ee880a207aae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:12:39.863921 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967888b0-5de9-482e-948a-ee880a207aae admin admin] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:12:39.864200 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967888b0-5de9-482e-948a-ee880a207aae admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:12:39.865362 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-967888b0-5de9-482e-948a-ee880a207aae admin admin] PATH_INFO: `/v3/users` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:12:39.867830 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-967888b0-5de9-482e-948a-ee880a207aae admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:12:39.870318 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-967888b0-5de9-482e-948a-ee880a207aae admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:12:39.933240 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8aa546-e850-417b-bd05-1884201844af admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:12:39.938480 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 885/1765] 10.222.0.52 () {60 vars in 1143 bytes} [Wed Jul 13 22:12:39 2022] POST /identity/v3/users => generated 311 bytes in 324 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 13 22:12:39.948831 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8aa546-e850-417b-bd05-1884201844af admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:12:39.955610 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8363c8-3fd9-43ce-8e52-7101db9b65cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:12:39.957410 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8363c8-3fd9-43ce-8e52-7101db9b65cb admin admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:12:39.957955 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8363c8-3fd9-43ce-8e52-7101db9b65cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:12:39.958603 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b8363c8-3fd9-43ce-8e52-7101db9b65cb admin admin] PATH_INFO: `/v3/roles` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:12:39.959853 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b8363c8-3fd9-43ce-8e52-7101db9b65cb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:12:39.961941 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b8363c8-3fd9-43ce-8e52-7101db9b65cb admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:12:39.978913 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 881/1766] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jul 13 22:12:39 2022] GET /identity/v3/roles => generated 1420 bytes in 46 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 22:12:39.985597 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967888b0-5de9-482e-948a-ee880a207aae admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:12:40.003233 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-967888b0-5de9-482e-948a-ee880a207aae admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:12:40.019328 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44487cab-9c7d-4635-887d-62ff98ab3c3d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:12:40.023353 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44487cab-9c7d-4635-887d-62ff98ab3c3d admin admin] REQUEST_METHOD: `PUT` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:12:40.024353 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44487cab-9c7d-4635-887d-62ff98ab3c3d admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:12:40.028314 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44487cab-9c7d-4635-887d-62ff98ab3c3d admin admin] PATH_INFO: `/v3/projects/416c136037734f6f938896f1e48cef3a/users/c2e80d88615e40309fa001d9baec709f/roles/44efba2579f342df8a0789c59d163ec3` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:12:40.105939 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44487cab-9c7d-4635-887d-62ff98ab3c3d admin admin] RBAC: Authorizing `identity:create_grant(project_id=416c136037734f6f938896f1e48cef3a, user_id=c2e80d88615e40309fa001d9baec709f, role_id=44efba2579f342df8a0789c59d163ec3)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:12:40.110126 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44487cab-9c7d-4635-887d-62ff98ab3c3d admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:12:40.169004 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 886/1767] 10.222.0.52 () {60 vars in 1482 bytes} [Wed Jul 13 22:12:39 2022] PUT /identity/v3/projects/416c136037734f6f938896f1e48cef3a/users/c2e80d88615e40309fa001d9baec709f/roles/44efba2579f342df8a0789c59d163ec3 => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 22:12:40.170261 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8363c8-3fd9-43ce-8e52-7101db9b65cb admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:12:40.191981 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b8363c8-3fd9-43ce-8e52-7101db9b65cb admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:12:40.422136 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9927d7b4-8103-4525-a351-ce9f3cf3ae68 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:12:40.423549 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9927d7b4-8103-4525-a351-ce9f3cf3ae68 admin admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:12:40.424001 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9927d7b4-8103-4525-a351-ce9f3cf3ae68 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:12:40.424533 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9927d7b4-8103-4525-a351-ce9f3cf3ae68 admin admin] PATH_INFO: `/v3/roles` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:12:40.425856 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9927d7b4-8103-4525-a351-ce9f3cf3ae68 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:12:40.432307 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9927d7b4-8103-4525-a351-ce9f3cf3ae68 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:12:40.465091 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 882/1768] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jul 13 22:12:40 2022] GET /identity/v3/roles => generated 1420 bytes in 295 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 22:12:40.469544 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44487cab-9c7d-4635-887d-62ff98ab3c3d admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:12:40.522855 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44487cab-9c7d-4635-887d-62ff98ab3c3d admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:12:40.535518 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b3d7e9-b360-435b-9052-ac4ec658524a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:12:40.556738 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b3d7e9-b360-435b-9052-ac4ec658524a admin admin] REQUEST_METHOD: `PUT` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:12:40.557234 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b3d7e9-b360-435b-9052-ac4ec658524a admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:12:40.557527 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9b3d7e9-b360-435b-9052-ac4ec658524a admin admin] PATH_INFO: `/v3/projects/416c136037734f6f938896f1e48cef3a/users/c2e80d88615e40309fa001d9baec709f/roles/44efba2579f342df8a0789c59d163ec3` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:12:40.563350 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b3d7e9-b360-435b-9052-ac4ec658524a admin admin] RBAC: Authorizing `identity:create_grant(project_id=416c136037734f6f938896f1e48cef3a, user_id=c2e80d88615e40309fa001d9baec709f, role_id=44efba2579f342df8a0789c59d163ec3)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:12:40.568262 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9b3d7e9-b360-435b-9052-ac4ec658524a admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:12:40.588369 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 887/1769] 10.222.0.52 () {60 vars in 1482 bytes} [Wed Jul 13 22:12:40 2022] PUT /identity/v3/projects/416c136037734f6f938896f1e48cef3a/users/c2e80d88615e40309fa001d9baec709f/roles/44efba2579f342df8a0789c59d163ec3 => generated 0 bytes in 119 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 22:12:40.601025 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab06cc6-6b75-46b4-b0ac-3723526070df None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:12:40.601249 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab06cc6-6b75-46b4-b0ac-3723526070df None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:12:40.601597 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ab06cc6-6b75-46b4-b0ac-3723526070df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:12:40.697907 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-3ab06cc6-6b75-46b4-b0ac-3723526070df None None] MFA Rules not processed for user `c2e80d88615e40309fa001d9baec709f`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 22:12:40.699812 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-3ab06cc6-6b75-46b4-b0ac-3723526070df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:12:40.851793 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 883/1770] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jul 13 22:12:40 2022] POST /identity/v3/auth/tokens => generated 3343 bytes in 249 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 13 22:12:40.868238 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b3d7e9-b360-435b-9052-ac4ec658524a admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:12:40.870546 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-d9b3d7e9-b360-435b-9052-ac4ec658524a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:12:40.939249 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9b3d7e9-b360-435b-9052-ac4ec658524a admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:12:40.965248 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feaea017-072b-48fa-bfbd-5b716b502e90 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc77e1ea1ada4700b809beb1c980c5e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417e3e2f1333481795a089bef2e24bea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:12:40.989528 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feaea017-072b-48fa-bfbd-5b716b502e90 service cinder] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:12:40.989818 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feaea017-072b-48fa-bfbd-5b716b502e90 service cinder] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:12:40.990097 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-feaea017-072b-48fa-bfbd-5b716b502e90 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:12:41.017518 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feaea017-072b-48fa-bfbd-5b716b502e90 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:12:41.019130 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-feaea017-072b-48fa-bfbd-5b716b502e90 service cinder] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:12:41.031543 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 888/1771] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jul 13 22:12:40 2022] GET /identity/v3/auth/tokens => generated 3343 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 22:12:41.229685 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28d2a76-71e9-4097-8bde-c4323079aab4 None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:12:41.230250 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28d2a76-71e9-4097-8bde-c4323079aab4 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:12:41.230355 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b28d2a76-71e9-4097-8bde-c4323079aab4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:12:41.263225 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-b28d2a76-71e9-4097-8bde-c4323079aab4 None None] MFA Rules not processed for user `064dcac8f59c431cb7d6848f4aaac347`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 22:12:41.265707 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-b28d2a76-71e9-4097-8bde-c4323079aab4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:12:41.334419 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:12:41.369428 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feaea017-072b-48fa-bfbd-5b716b502e90 service cinder] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:12:41.375038 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 884/1772] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jul 13 22:12:41 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 146 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 22:12:41.379750 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-feaea017-072b-48fa-bfbd-5b716b502e90 service cinder] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:12:41.385902 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3070eff8-a945-40ed-a79c-f203afde3b7d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc77e1ea1ada4700b809beb1c980c5e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417e3e2f1333481795a089bef2e24bea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:12:41.386622 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3070eff8-a945-40ed-a79c-f203afde3b7d service cinder] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:12:41.387065 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3070eff8-a945-40ed-a79c-f203afde3b7d service cinder] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:12:41.387467 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3070eff8-a945-40ed-a79c-f203afde3b7d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:12:41.515641 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3070eff8-a945-40ed-a79c-f203afde3b7d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:12:41.517126 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3070eff8-a945-40ed-a79c-f203afde3b7d service cinder] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:12:41.530791 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 889/1773] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jul 13 22:12:41 2022] GET /identity/v3/auth/tokens => generated 3276 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 13 22:12:51.350364 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=4 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:12:51.525460 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=80,keystone:INSERT=5 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:12:53.718647 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a410de2c-5baa-4d6e-bfe7-1e7a2a8b64bf None None] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:12:53.719170 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a410de2c-5baa-4d6e-bfe7-1e7a2a8b64bf None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:12:53.719332 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a410de2c-5baa-4d6e-bfe7-1e7a2a8b64bf None None] PATH_INFO: `/` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:12:53.722054 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 885/1774] 10.222.0.52 () {54 vars in 921 bytes} [Wed Jul 13 22:12:53 2022] GET /identity => generated 270 bytes in 5 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 22:12:53.738071 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699bc79b-d82d-4f80-9b99-504f44803ffb None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:12:53.738744 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699bc79b-d82d-4f80-9b99-504f44803ffb None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:12:53.738744 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-699bc79b-d82d-4f80-9b99-504f44803ffb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:12:53.763580 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-699bc79b-d82d-4f80-9b99-504f44803ffb None None] MFA Rules not processed for user `f84281091687464990ffbca590b9343b`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 22:12:53.772869 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-699bc79b-d82d-4f80-9b99-504f44803ffb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:12:53.938413 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 890/1775] 10.222.0.52 () {58 vars in 1018 bytes} [Wed Jul 13 22:12:53 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 22:12:53.951608 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a410de2c-5baa-4d6e-bfe7-1e7a2a8b64bf None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:12:53.953493 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-a410de2c-5baa-4d6e-bfe7-1e7a2a8b64bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:12:53.981782 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a410de2c-5baa-4d6e-bfe7-1e7a2a8b64bf None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:12:53.989978 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a46415-5547-4cef-baa3-0567c6ee8b4c service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84281091687464990ffbca590b9343b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417e3e2f1333481795a089bef2e24bea', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:12:53.990692 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a46415-5547-4cef-baa3-0567c6ee8b4c service nova] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:12:53.991085 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a46415-5547-4cef-baa3-0567c6ee8b4c service nova] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:12:53.991369 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a46415-5547-4cef-baa3-0567c6ee8b4c service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:12:54.001386 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a46415-5547-4cef-baa3-0567c6ee8b4c service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:12:54.003117 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a46415-5547-4cef-baa3-0567c6ee8b4c service nova] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:12:54.016059 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 886/1776] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jul 13 22:12:53 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 65 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 22:12:55.074923 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699bc79b-d82d-4f80-9b99-504f44803ffb None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:12:55.076434 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-699bc79b-d82d-4f80-9b99-504f44803ffb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:12:55.144328 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-699bc79b-d82d-4f80-9b99-504f44803ffb None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:12:55.149418 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27855913-03cf-4c9a-8941-bf0c10d33d78 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:12:55.150319 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27855913-03cf-4c9a-8941-bf0c10d33d78 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:12:55.150652 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27855913-03cf-4c9a-8941-bf0c10d33d78 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:12:55.150927 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27855913-03cf-4c9a-8941-bf0c10d33d78 admin admin] PATH_INFO: `/v3/users/e7437c0da1864553b85d369b65b768bd` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:12:55.152264 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27855913-03cf-4c9a-8941-bf0c10d33d78 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e7437c0da1864553b85d369b65b768bd)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:12:55.154225 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27855913-03cf-4c9a-8941-bf0c10d33d78 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:12:55.212604 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-27855913-03cf-4c9a-8941-bf0c10d33d78 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e7437c0da1864553b85d369b65b768bd'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:12:55.222257 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-27855913-03cf-4c9a-8941-bf0c10d33d78 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e7437c0da1864553b85d369b65b768bd'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:12:55.235287 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-27855913-03cf-4c9a-8941-bf0c10d33d78 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e7437c0da1864553b85d369b65b768bd'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:12:55.236096 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-27855913-03cf-4c9a-8941-bf0c10d33d78 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e7437c0da1864553b85d369b65b768bd'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:12:55.236806 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-27855913-03cf-4c9a-8941-bf0c10d33d78 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e7437c0da1864553b85d369b65b768bd'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:12:55.249741 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 891/1777] 10.222.0.52 () {58 vars in 1223 bytes} [Wed Jul 13 22:12:55 2022] DELETE /identity/v3/users/e7437c0da1864553b85d369b65b768bd => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 22:12:55.254427 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18248ebd-d6e0-4f36-b5f7-024f09d21c47 None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:12:55.254976 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18248ebd-d6e0-4f36-b5f7-024f09d21c47 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:12:55.255084 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18248ebd-d6e0-4f36-b5f7-024f09d21c47 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:12:55.272972 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-18248ebd-d6e0-4f36-b5f7-024f09d21c47 None None] MFA Rules not processed for user `cc99c93da5074cf5a48bc2aa9a92c5e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 22:12:55.274314 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-18248ebd-d6e0-4f36-b5f7-024f09d21c47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:12:55.334895 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 887/1778] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jul 13 22:12:55 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 22:12:55.344871 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27855913-03cf-4c9a-8941-bf0c10d33d78 admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:12:55.346384 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-27855913-03cf-4c9a-8941-bf0c10d33d78 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:12:55.361632 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:12:55.409062 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27855913-03cf-4c9a-8941-bf0c10d33d78 admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:12:55.412978 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312dcd66-8358-4b93-8286-e7e49d96932c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '847ec9a1bd4d4dec804318a5c845895a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417e3e2f1333481795a089bef2e24bea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:12:55.413664 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312dcd66-8358-4b93-8286-e7e49d96932c service neutron] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:12:55.414144 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312dcd66-8358-4b93-8286-e7e49d96932c service neutron] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:12:55.414240 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-312dcd66-8358-4b93-8286-e7e49d96932c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:12:55.424685 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312dcd66-8358-4b93-8286-e7e49d96932c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:12:55.427099 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-312dcd66-8358-4b93-8286-e7e49d96932c service neutron] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:12:55.440300 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 892/1779] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jul 13 22:12:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 22:12:55.747668 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18248ebd-d6e0-4f36-b5f7-024f09d21c47 None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:12:55.749408 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-18248ebd-d6e0-4f36-b5f7-024f09d21c47 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:12:55.762340 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18248ebd-d6e0-4f36-b5f7-024f09d21c47 None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:12:55.767564 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595cf410-073e-4b74-a699-f8b061d2e4ab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:12:55.768438 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595cf410-073e-4b74-a699-f8b061d2e4ab admin admin] REQUEST_METHOD: `DELETE` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:12:55.769063 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595cf410-073e-4b74-a699-f8b061d2e4ab admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:12:55.769354 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-595cf410-073e-4b74-a699-f8b061d2e4ab admin admin] PATH_INFO: `/v3/projects/4e3d7c10ff3841019119ef8b83381fe1` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:12:55.770738 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595cf410-073e-4b74-a699-f8b061d2e4ab admin admin] RBAC: Authorizing `identity:delete_project(project_id=4e3d7c10ff3841019119ef8b83381fe1)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:12:55.772983 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-595cf410-073e-4b74-a699-f8b061d2e4ab admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:12:55.824724 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-595cf410-073e-4b74-a699-f8b061d2e4ab admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4e3d7c10ff3841019119ef8b83381fe1', 'request_id': 'req-595cf410-073e-4b74-a699-f8b061d2e4ab'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:12:55.834993 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-595cf410-073e-4b74-a699-f8b061d2e4ab admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4e3d7c10ff3841019119ef8b83381fe1', 'request_id': 'req-595cf410-073e-4b74-a699-f8b061d2e4ab'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:12:55.842149 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-595cf410-073e-4b74-a699-f8b061d2e4ab admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4e3d7c10ff3841019119ef8b83381fe1', 'request_id': 'req-595cf410-073e-4b74-a699-f8b061d2e4ab'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:12:55.859383 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-595cf410-073e-4b74-a699-f8b061d2e4ab admin admin] The token cache is being invalidate because project 4e3d7c10ff3841019119ef8b83381fe1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80382) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 22:12:55.859727 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-595cf410-073e-4b74-a699-f8b061d2e4ab admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:12:55.860329 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-595cf410-073e-4b74-a699-f8b061d2e4ab admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:12:55.862126 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 888/1780] 10.222.0.52 () {58 vars in 1232 bytes} [Wed Jul 13 22:12:55 2022] DELETE /identity/v3/projects/4e3d7c10ff3841019119ef8b83381fe1 => generated 0 bytes in 115 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 22:12:55.866195 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312dcd66-8358-4b93-8286-e7e49d96932c service neutron] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:12:55.882538 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-312dcd66-8358-4b93-8286-e7e49d96932c service neutron] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:12:55.883633 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595cf410-073e-4b74-a699-f8b061d2e4ab admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:12:55.885037 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-595cf410-073e-4b74-a699-f8b061d2e4ab admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:12:55.997170 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d460b9c3-d59e-435c-a893-69cb03ab0c3e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:12:55.998472 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d460b9c3-d59e-435c-a893-69cb03ab0c3e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:12:56.000062 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d460b9c3-d59e-435c-a893-69cb03ab0c3e admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:12:56.000598 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d460b9c3-d59e-435c-a893-69cb03ab0c3e admin admin] PATH_INFO: `/v3/users/06bdd92ae98c4620b3b400486eb55bdf` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:12:56.001987 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d460b9c3-d59e-435c-a893-69cb03ab0c3e admin admin] RBAC: Authorizing `identity:delete_user(user_id=06bdd92ae98c4620b3b400486eb55bdf)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:12:56.004240 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d460b9c3-d59e-435c-a893-69cb03ab0c3e admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:12:56.023730 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-595cf410-073e-4b74-a699-f8b061d2e4ab admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:12:56.027481 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9dbdf21-5c13-4a34-ac50-57a6a35b6256 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:12:56.028583 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9dbdf21-5c13-4a34-ac50-57a6a35b6256 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:12:56.028816 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9dbdf21-5c13-4a34-ac50-57a6a35b6256 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:12:56.029088 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9dbdf21-5c13-4a34-ac50-57a6a35b6256 admin admin] PATH_INFO: `/v3/users/e709cc7e6bdc48f2842d69889e2b0c1f` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:12:56.030522 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9dbdf21-5c13-4a34-ac50-57a6a35b6256 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e709cc7e6bdc48f2842d69889e2b0c1f)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:12:56.032730 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9dbdf21-5c13-4a34-ac50-57a6a35b6256 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:12:56.048372 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:12:56.076888 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-d460b9c3-d59e-435c-a893-69cb03ab0c3e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '06bdd92ae98c4620b3b400486eb55bdf'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:12:56.084225 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-d460b9c3-d59e-435c-a893-69cb03ab0c3e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '06bdd92ae98c4620b3b400486eb55bdf'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:12:56.092082 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-b9dbdf21-5c13-4a34-ac50-57a6a35b6256 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e709cc7e6bdc48f2842d69889e2b0c1f'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:12:56.099278 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-d460b9c3-d59e-435c-a893-69cb03ab0c3e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '06bdd92ae98c4620b3b400486eb55bdf'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:12:56.099901 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-d460b9c3-d59e-435c-a893-69cb03ab0c3e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '06bdd92ae98c4620b3b400486eb55bdf'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:12:56.100513 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-d460b9c3-d59e-435c-a893-69cb03ab0c3e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '06bdd92ae98c4620b3b400486eb55bdf'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:12:56.100737 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-b9dbdf21-5c13-4a34-ac50-57a6a35b6256 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e709cc7e6bdc48f2842d69889e2b0c1f'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:12:56.114757 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-b9dbdf21-5c13-4a34-ac50-57a6a35b6256 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e709cc7e6bdc48f2842d69889e2b0c1f'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:12:56.115519 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-b9dbdf21-5c13-4a34-ac50-57a6a35b6256 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e709cc7e6bdc48f2842d69889e2b0c1f'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:12:56.117191 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 893/1781] 10.222.0.52 () {58 vars in 1223 bytes} [Wed Jul 13 22:12:55 2022] DELETE /identity/v3/users/06bdd92ae98c4620b3b400486eb55bdf => generated 0 bytes in 252 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 13 22:12:56.117532 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-b9dbdf21-5c13-4a34-ac50-57a6a35b6256 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e709cc7e6bdc48f2842d69889e2b0c1f'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:12:56.133622 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 889/1782] 10.222.0.52 () {58 vars in 1223 bytes} [Wed Jul 13 22:12:55 2022] DELETE /identity/v3/users/e709cc7e6bdc48f2842d69889e2b0c1f => generated 0 bytes in 251 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 13 22:12:56.138409 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec74f81-9d76-426a-96dd-c52988cc4fd4 None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:12:56.138726 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec74f81-9d76-426a-96dd-c52988cc4fd4 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:12:56.139337 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ec74f81-9d76-426a-96dd-c52988cc4fd4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:12:56.157288 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-2ec74f81-9d76-426a-96dd-c52988cc4fd4 None None] MFA Rules not processed for user `cc99c93da5074cf5a48bc2aa9a92c5e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 22:12:56.158438 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-2ec74f81-9d76-426a-96dd-c52988cc4fd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:12:56.212856 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:12:56.259555 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 894/1783] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jul 13 22:12:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 22:12:56.266873 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9dbdf21-5c13-4a34-ac50-57a6a35b6256 admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:12:56.268390 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-b9dbdf21-5c13-4a34-ac50-57a6a35b6256 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:12:56.343662 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9dbdf21-5c13-4a34-ac50-57a6a35b6256 admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:12:56.348072 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51b4d60-24e3-4730-9bc2-007d4a8c5e03 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '847ec9a1bd4d4dec804318a5c845895a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417e3e2f1333481795a089bef2e24bea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:12:56.349076 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51b4d60-24e3-4730-9bc2-007d4a8c5e03 service neutron] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:12:56.350600 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51b4d60-24e3-4730-9bc2-007d4a8c5e03 service neutron] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:12:56.350600 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51b4d60-24e3-4730-9bc2-007d4a8c5e03 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:12:56.367838 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51b4d60-24e3-4730-9bc2-007d4a8c5e03 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:12:56.369493 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b51b4d60-24e3-4730-9bc2-007d4a8c5e03 service neutron] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:12:56.395769 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 890/1784] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jul 13 22:12:56 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 129 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 22:12:56.650803 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec74f81-9d76-426a-96dd-c52988cc4fd4 None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:12:56.652750 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-2ec74f81-9d76-426a-96dd-c52988cc4fd4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:12:56.669182 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ec74f81-9d76-426a-96dd-c52988cc4fd4 None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:12:56.688927 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39df4e66-7dc6-42a4-89b8-f3e75610b13a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:12:56.690255 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39df4e66-7dc6-42a4-89b8-f3e75610b13a admin admin] REQUEST_METHOD: `DELETE` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:12:56.690679 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39df4e66-7dc6-42a4-89b8-f3e75610b13a admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:12:56.691057 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39df4e66-7dc6-42a4-89b8-f3e75610b13a admin admin] PATH_INFO: `/v3/projects/4b382a3b903940068088455f2abbc792` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:12:56.692482 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39df4e66-7dc6-42a4-89b8-f3e75610b13a admin admin] RBAC: Authorizing `identity:delete_project(project_id=4b382a3b903940068088455f2abbc792)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:12:56.694768 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39df4e66-7dc6-42a4-89b8-f3e75610b13a admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:12:56.772697 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-39df4e66-7dc6-42a4-89b8-f3e75610b13a admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4b382a3b903940068088455f2abbc792', 'request_id': 'req-39df4e66-7dc6-42a4-89b8-f3e75610b13a'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:12:56.782704 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-39df4e66-7dc6-42a4-89b8-f3e75610b13a admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4b382a3b903940068088455f2abbc792', 'request_id': 'req-39df4e66-7dc6-42a4-89b8-f3e75610b13a'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:12:56.795284 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-39df4e66-7dc6-42a4-89b8-f3e75610b13a admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4b382a3b903940068088455f2abbc792', 'request_id': 'req-39df4e66-7dc6-42a4-89b8-f3e75610b13a'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:12:56.810888 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-39df4e66-7dc6-42a4-89b8-f3e75610b13a admin admin] The token cache is being invalidate because project 4b382a3b903940068088455f2abbc792 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80381) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 22:12:56.811414 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-39df4e66-7dc6-42a4-89b8-f3e75610b13a admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:12:56.812073 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-39df4e66-7dc6-42a4-89b8-f3e75610b13a admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:12:56.813782 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 895/1785] 10.222.0.52 () {58 vars in 1232 bytes} [Wed Jul 13 22:12:56 2022] DELETE /identity/v3/projects/4b382a3b903940068088455f2abbc792 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 22:12:56.818952 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76dad028-f80a-4a21-86ac-141e8e2e1374 None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:12:56.819271 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76dad028-f80a-4a21-86ac-141e8e2e1374 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:12:56.819553 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76dad028-f80a-4a21-86ac-141e8e2e1374 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:12:56.839697 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-76dad028-f80a-4a21-86ac-141e8e2e1374 None None] MFA Rules not processed for user `cc99c93da5074cf5a48bc2aa9a92c5e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 22:12:56.840988 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-76dad028-f80a-4a21-86ac-141e8e2e1374 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:12:56.867822 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39df4e66-7dc6-42a4-89b8-f3e75610b13a admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:12:56.869470 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-39df4e66-7dc6-42a4-89b8-f3e75610b13a admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:12:56.906378 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 891/1786] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jul 13 22:12:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 22:12:56.914228 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab0647a-f58b-41ef-b0d8-99cf59e85f30 None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:12:56.914543 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab0647a-f58b-41ef-b0d8-99cf59e85f30 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:12:56.914828 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ab0647a-f58b-41ef-b0d8-99cf59e85f30 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:12:56.939742 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-4ab0647a-f58b-41ef-b0d8-99cf59e85f30 None None] MFA Rules not processed for user `cc99c93da5074cf5a48bc2aa9a92c5e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 22:12:56.941038 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-4ab0647a-f58b-41ef-b0d8-99cf59e85f30 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:12:56.944605 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39df4e66-7dc6-42a4-89b8-f3e75610b13a admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:12:56.952601 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f23c7f8-acc4-48b9-8b89-8972a9a2e1c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:12:56.953518 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 892/1787] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jul 13 22:12:56 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 40 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 13 22:12:56.953982 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f23c7f8-acc4-48b9-8b89-8972a9a2e1c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:12:56.954268 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f23c7f8-acc4-48b9-8b89-8972a9a2e1c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:12:56.954747 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f23c7f8-acc4-48b9-8b89-8972a9a2e1c4 admin admin] PATH_INFO: `/v3/projects/265a530513fd48bdaa63754a18cfb047` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:12:56.956598 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab0647a-f58b-41ef-b0d8-99cf59e85f30 None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:12:56.958054 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f23c7f8-acc4-48b9-8b89-8972a9a2e1c4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=265a530513fd48bdaa63754a18cfb047)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:12:56.960014 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2f23c7f8-acc4-48b9-8b89-8972a9a2e1c4 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:12:56.969155 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ab0647a-f58b-41ef-b0d8-99cf59e85f30 None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:12:56.975733 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1358ffd5-f6bd-4610-9f93-9ebe04e5bb72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:12:56.976988 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1358ffd5-f6bd-4610-9f93-9ebe04e5bb72 admin admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:12:56.977275 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1358ffd5-f6bd-4610-9f93-9ebe04e5bb72 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:12:56.977556 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1358ffd5-f6bd-4610-9f93-9ebe04e5bb72 admin admin] PATH_INFO: `/v3/domains` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:12:56.978767 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1358ffd5-f6bd-4610-9f93-9ebe04e5bb72 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 22:12:57.002404 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1358ffd5-f6bd-4610-9f93-9ebe04e5bb72 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:12:57.004300 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1358ffd5-f6bd-4610-9f93-9ebe04e5bb72 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:12:57.023751 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-2f23c7f8-acc4-48b9-8b89-8972a9a2e1c4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '265a530513fd48bdaa63754a18cfb047', 'request_id': 'req-2f23c7f8-acc4-48b9-8b89-8972a9a2e1c4'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:12:57.037814 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-2f23c7f8-acc4-48b9-8b89-8972a9a2e1c4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '265a530513fd48bdaa63754a18cfb047', 'request_id': 'req-2f23c7f8-acc4-48b9-8b89-8972a9a2e1c4'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:12:57.056759 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:12:57.062710 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-2f23c7f8-acc4-48b9-8b89-8972a9a2e1c4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '265a530513fd48bdaa63754a18cfb047', 'request_id': 'req-2f23c7f8-acc4-48b9-8b89-8972a9a2e1c4'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:12:57.073428 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 893/1788] 10.222.0.52 () {58 vars in 1152 bytes} [Wed Jul 13 22:12:56 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 117 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 22:12:57.080224 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1358ffd5-f6bd-4610-9f93-9ebe04e5bb72 admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:12:57.108472 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-2f23c7f8-acc4-48b9-8b89-8972a9a2e1c4 admin admin] The token cache is being invalidate because project 265a530513fd48bdaa63754a18cfb047 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80381) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 22:12:57.109036 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-2f23c7f8-acc4-48b9-8b89-8972a9a2e1c4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:12:57.110326 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-2f23c7f8-acc4-48b9-8b89-8972a9a2e1c4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:12:57.110448 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1358ffd5-f6bd-4610-9f93-9ebe04e5bb72 admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:12:57.113583 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 896/1789] 10.222.0.52 () {58 vars in 1232 bytes} [Wed Jul 13 22:12:56 2022] DELETE /identity/v3/projects/265a530513fd48bdaa63754a18cfb047 => generated 0 bytes in 247 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 13 22:12:57.123309 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f23c7f8-acc4-48b9-8b89-8972a9a2e1c4 admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:12:57.126006 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-2f23c7f8-acc4-48b9-8b89-8972a9a2e1c4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:12:57.128979 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=90,keystone:DELETE=9,keystone:INSERT=1 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:12:57.214449 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5783f09-708e-4b70-998d-4859300dc0c5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:12:57.215308 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5783f09-708e-4b70-998d-4859300dc0c5 admin admin] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:12:57.215588 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5783f09-708e-4b70-998d-4859300dc0c5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:12:57.215891 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5783f09-708e-4b70-998d-4859300dc0c5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:12:57.217609 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5783f09-708e-4b70-998d-4859300dc0c5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:12:57.219802 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5783f09-708e-4b70-998d-4859300dc0c5 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:12:57.315343 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 894/1790] 10.222.0.52 () {60 vars in 1152 bytes} [Wed Jul 13 22:12:57 2022] POST /identity/v3/projects => generated 369 bytes in 238 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 22:12:57.318994 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5783f09-708e-4b70-998d-4859300dc0c5 admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:12:57.322942 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-a5783f09-708e-4b70-998d-4859300dc0c5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:12:57.336675 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f23c7f8-acc4-48b9-8b89-8972a9a2e1c4 admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:12:57.496700 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f7dc733-db83-4397-b58c-b343f074735e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:12:57.497620 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7dc733-db83-4397-b58c-b343f074735e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:12:57.497910 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7dc733-db83-4397-b58c-b343f074735e admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:12:57.498190 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f7dc733-db83-4397-b58c-b343f074735e admin admin] PATH_INFO: `/v3/users/c2e80d88615e40309fa001d9baec709f` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:12:57.500537 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5783f09-708e-4b70-998d-4859300dc0c5 admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:12:57.505292 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f7dc733-db83-4397-b58c-b343f074735e admin admin] RBAC: Authorizing `identity:delete_user(user_id=c2e80d88615e40309fa001d9baec709f)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:12:57.508903 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697c7db7-becf-4a46-9bdb-71e5d75e4735 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:12:57.509766 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697c7db7-becf-4a46-9bdb-71e5d75e4735 admin admin] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:12:57.510246 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697c7db7-becf-4a46-9bdb-71e5d75e4735 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:12:57.510832 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-697c7db7-becf-4a46-9bdb-71e5d75e4735 admin admin] PATH_INFO: `/v3/users` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:12:57.511732 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-697c7db7-becf-4a46-9bdb-71e5d75e4735 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:12:57.513080 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f7dc733-db83-4397-b58c-b343f074735e admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:12:57.515799 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-697c7db7-becf-4a46-9bdb-71e5d75e4735 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:12:57.572800 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 895/1791] 10.222.0.52 () {60 vars in 1143 bytes} [Wed Jul 13 22:12:57 2022] POST /identity/v3/users => generated 285 bytes in 251 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 22:12:57.574339 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697c7db7-becf-4a46-9bdb-71e5d75e4735 admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:12:57.592880 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-697c7db7-becf-4a46-9bdb-71e5d75e4735 admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:12:57.604737 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ff4367-bc15-45da-be72-952855b0487e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:12:57.605957 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ff4367-bc15-45da-be72-952855b0487e admin admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:12:57.606473 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ff4367-bc15-45da-be72-952855b0487e admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:12:57.606804 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ff4367-bc15-45da-be72-952855b0487e admin admin] PATH_INFO: `/v3/roles` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:12:57.610175 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ff4367-bc15-45da-be72-952855b0487e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:12:57.613634 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ff4367-bc15-45da-be72-952855b0487e admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:12:57.632852 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 896/1792] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jul 13 22:12:57 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 13 22:12:57.637664 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ff4367-bc15-45da-be72-952855b0487e admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:12:57.656120 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ff4367-bc15-45da-be72-952855b0487e admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:12:57.663526 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4def537a-2ab1-4565-aa48-fdf193bcc16f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:12:57.664353 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4def537a-2ab1-4565-aa48-fdf193bcc16f admin admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:12:57.664728 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4def537a-2ab1-4565-aa48-fdf193bcc16f admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:12:57.665006 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-9f7dc733-db83-4397-b58c-b343f074735e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c2e80d88615e40309fa001d9baec709f'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:12:57.665120 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4def537a-2ab1-4565-aa48-fdf193bcc16f admin admin] PATH_INFO: `/v3/roles` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:12:57.667580 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4def537a-2ab1-4565-aa48-fdf193bcc16f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:12:57.669935 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4def537a-2ab1-4565-aa48-fdf193bcc16f admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:12:57.688685 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-9f7dc733-db83-4397-b58c-b343f074735e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c2e80d88615e40309fa001d9baec709f'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:12:57.693027 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 897/1793] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jul 13 22:12:57 2022] GET /identity/v3/roles => generated 1420 bytes in 55 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 22:12:57.699618 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4def537a-2ab1-4565-aa48-fdf193bcc16f admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:12:57.708997 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-9f7dc733-db83-4397-b58c-b343f074735e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c2e80d88615e40309fa001d9baec709f'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:12:57.710133 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-9f7dc733-db83-4397-b58c-b343f074735e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c2e80d88615e40309fa001d9baec709f'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:12:57.711033 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-9f7dc733-db83-4397-b58c-b343f074735e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c2e80d88615e40309fa001d9baec709f'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:12:57.728493 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4def537a-2ab1-4565-aa48-fdf193bcc16f admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:12:57.734976 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8602815a-5711-49e8-8470-6a6734dbc606 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:12:57.735948 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8602815a-5711-49e8-8470-6a6734dbc606 admin admin] REQUEST_METHOD: `PUT` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:12:57.736234 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8602815a-5711-49e8-8470-6a6734dbc606 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:12:57.736517 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8602815a-5711-49e8-8470-6a6734dbc606 admin admin] PATH_INFO: `/v3/projects/4ca8f8a653e44e50a620ba8b9401b574/users/91d8566c5b0940b9ab7d0efea689839e/roles/2a5e86976394498abb6310697cdb3f8b` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:12:57.752329 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 897/1794] 10.222.0.52 () {58 vars in 1223 bytes} [Wed Jul 13 22:12:57 2022] DELETE /identity/v3/users/c2e80d88615e40309fa001d9baec709f => generated 0 bytes in 634 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 22:12:57.758551 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8602815a-5711-49e8-8470-6a6734dbc606 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4ca8f8a653e44e50a620ba8b9401b574, user_id=91d8566c5b0940b9ab7d0efea689839e, role_id=2a5e86976394498abb6310697cdb3f8b)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:12:57.761265 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8602815a-5711-49e8-8470-6a6734dbc606 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:12:57.789688 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 898/1795] 10.222.0.52 () {60 vars in 1482 bytes} [Wed Jul 13 22:12:57 2022] PUT /identity/v3/projects/4ca8f8a653e44e50a620ba8b9401b574/users/91d8566c5b0940b9ab7d0efea689839e/roles/2a5e86976394498abb6310697cdb3f8b => generated 0 bytes in 80 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 22:12:57.792589 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106f7abc-afc7-4112-bd72-e51618ac4e3b None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:12:57.793120 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106f7abc-afc7-4112-bd72-e51618ac4e3b None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:12:57.793616 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-106f7abc-afc7-4112-bd72-e51618ac4e3b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:12:57.845603 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:DELETE=12,keystone:SELECT=86,keystone:INSERT=2 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:12:57.849394 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-106f7abc-afc7-4112-bd72-e51618ac4e3b None None] MFA Rules not processed for user `91d8566c5b0940b9ab7d0efea689839e`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 22:12:57.851795 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-106f7abc-afc7-4112-bd72-e51618ac4e3b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:12:57.985674 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 898/1796] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jul 13 22:12:57 2022] POST /identity/v3/auth/tokens => generated 3236 bytes in 194 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 22:12:57.988066 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8602815a-5711-49e8-8470-6a6734dbc606 admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:12:57.991345 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-8602815a-5711-49e8-8470-6a6734dbc606 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:12:58.086120 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:12:58.121987 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8602815a-5711-49e8-8470-6a6734dbc606 admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:12:58.137848 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5153f39d-e1f4-4c3b-ab59-dabd6dbe3ced admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:12:58.139153 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5153f39d-e1f4-4c3b-ab59-dabd6dbe3ced admin admin] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:12:58.139153 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5153f39d-e1f4-4c3b-ab59-dabd6dbe3ced admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:12:58.139892 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5153f39d-e1f4-4c3b-ab59-dabd6dbe3ced admin admin] PATH_INFO: `/v3/projects` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:12:58.139999 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5153f39d-e1f4-4c3b-ab59-dabd6dbe3ced admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:12:58.141946 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5153f39d-e1f4-4c3b-ab59-dabd6dbe3ced admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:12:58.198967 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 899/1797] 10.222.0.52 () {60 vars in 1152 bytes} [Wed Jul 13 22:12:57 2022] POST /identity/v3/projects => generated 365 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 22:12:58.202765 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106f7abc-afc7-4112-bd72-e51618ac4e3b None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:12:58.218276 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-106f7abc-afc7-4112-bd72-e51618ac4e3b None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:12:58.356213 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59415f9-6f2c-4f95-9c22-8b4f25073b2b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:12:58.357092 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59415f9-6f2c-4f95-9c22-8b4f25073b2b admin admin] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:12:58.357473 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59415f9-6f2c-4f95-9c22-8b4f25073b2b admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:12:58.359099 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e59415f9-6f2c-4f95-9c22-8b4f25073b2b admin admin] PATH_INFO: `/v3/users` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:12:58.361180 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e59415f9-6f2c-4f95-9c22-8b4f25073b2b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:12:58.366822 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e59415f9-6f2c-4f95-9c22-8b4f25073b2b admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:12:58.412961 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 899/1798] 10.222.0.52 () {60 vars in 1143 bytes} [Wed Jul 13 22:12:58 2022] POST /identity/v3/users => generated 289 bytes in 211 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 13 22:12:58.417499 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5153f39d-e1f4-4c3b-ab59-dabd6dbe3ced admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:12:58.429446 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5153f39d-e1f4-4c3b-ab59-dabd6dbe3ced admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:12:58.435328 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5cca6b3-dc05-42b1-928c-aa1446cf49bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:12:58.436103 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cca6b3-dc05-42b1-928c-aa1446cf49bd admin admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:12:58.436296 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cca6b3-dc05-42b1-928c-aa1446cf49bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:12:58.436579 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5cca6b3-dc05-42b1-928c-aa1446cf49bd admin admin] PATH_INFO: `/v3/roles` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:12:58.438099 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5cca6b3-dc05-42b1-928c-aa1446cf49bd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:12:58.439823 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5cca6b3-dc05-42b1-928c-aa1446cf49bd admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:12:58.453177 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 900/1799] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jul 13 22:12:58 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 22:12:58.458179 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59415f9-6f2c-4f95-9c22-8b4f25073b2b admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:12:58.473204 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e59415f9-6f2c-4f95-9c22-8b4f25073b2b admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:12:58.478993 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8387309e-ce23-4f20-ad72-37d73b238f34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:12:58.480152 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8387309e-ce23-4f20-ad72-37d73b238f34 admin admin] REQUEST_METHOD: `PUT` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:12:58.480435 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8387309e-ce23-4f20-ad72-37d73b238f34 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:12:58.480713 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8387309e-ce23-4f20-ad72-37d73b238f34 admin admin] PATH_INFO: `/v3/projects/056dcb29d90146638679553ed5150214/users/7a0b3364444544558aa9f88b32883710/roles/44efba2579f342df8a0789c59d163ec3` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:12:58.501028 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8387309e-ce23-4f20-ad72-37d73b238f34 admin admin] RBAC: Authorizing `identity:create_grant(project_id=056dcb29d90146638679553ed5150214, user_id=7a0b3364444544558aa9f88b32883710, role_id=44efba2579f342df8a0789c59d163ec3)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:12:58.503855 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8387309e-ce23-4f20-ad72-37d73b238f34 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:12:58.526194 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 900/1800] 10.222.0.52 () {60 vars in 1482 bytes} [Wed Jul 13 22:12:58 2022] PUT /identity/v3/projects/056dcb29d90146638679553ed5150214/users/7a0b3364444544558aa9f88b32883710/roles/44efba2579f342df8a0789c59d163ec3 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 22:12:58.530345 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5cca6b3-dc05-42b1-928c-aa1446cf49bd admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:12:58.552170 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5cca6b3-dc05-42b1-928c-aa1446cf49bd admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:12:58.617567 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00963607-3b2c-496a-9892-fe9dd023d0f2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:12:58.619786 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00963607-3b2c-496a-9892-fe9dd023d0f2 admin admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:12:58.620083 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00963607-3b2c-496a-9892-fe9dd023d0f2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:12:58.620367 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00963607-3b2c-496a-9892-fe9dd023d0f2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:12:58.621123 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00963607-3b2c-496a-9892-fe9dd023d0f2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:12:58.622812 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00963607-3b2c-496a-9892-fe9dd023d0f2 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:12:58.635199 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 901/1801] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jul 13 22:12:58 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 22:12:58.639180 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8387309e-ce23-4f20-ad72-37d73b238f34 admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:12:58.665322 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8387309e-ce23-4f20-ad72-37d73b238f34 admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:12:58.681824 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca45041-2323-43cb-9515-0f983f211286 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:12:58.683950 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca45041-2323-43cb-9515-0f983f211286 admin admin] REQUEST_METHOD: `PUT` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:12:58.684289 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca45041-2323-43cb-9515-0f983f211286 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:12:58.684915 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ca45041-2323-43cb-9515-0f983f211286 admin admin] PATH_INFO: `/v3/projects/056dcb29d90146638679553ed5150214/users/7a0b3364444544558aa9f88b32883710/roles/44efba2579f342df8a0789c59d163ec3` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:12:58.687983 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca45041-2323-43cb-9515-0f983f211286 admin admin] RBAC: Authorizing `identity:create_grant(project_id=056dcb29d90146638679553ed5150214, user_id=7a0b3364444544558aa9f88b32883710, role_id=44efba2579f342df8a0789c59d163ec3)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:12:58.690750 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ca45041-2323-43cb-9515-0f983f211286 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:12:58.729386 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 901/1802] 10.222.0.52 () {60 vars in 1482 bytes} [Wed Jul 13 22:12:58 2022] PUT /identity/v3/projects/056dcb29d90146638679553ed5150214/users/7a0b3364444544558aa9f88b32883710/roles/44efba2579f342df8a0789c59d163ec3 => generated 0 bytes in 90 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 13 22:12:58.745482 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4297ec4d-2e11-4e73-befb-16029bf17fe5 None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:12:58.745983 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4297ec4d-2e11-4e73-befb-16029bf17fe5 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:12:58.746403 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4297ec4d-2e11-4e73-befb-16029bf17fe5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:12:58.778026 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca45041-2323-43cb-9515-0f983f211286 admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:12:58.779335 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-4297ec4d-2e11-4e73-befb-16029bf17fe5 None None] MFA Rules not processed for user `7a0b3364444544558aa9f88b32883710`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 22:12:58.781677 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-4297ec4d-2e11-4e73-befb-16029bf17fe5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:12:58.782721 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-6ca45041-2323-43cb-9515-0f983f211286 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:12:58.872828 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:12:58.881967 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ca45041-2323-43cb-9515-0f983f211286 admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:12:58.888123 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-682378f3-ea7b-4426-9184-0af9d8160bb0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:12:58.889014 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682378f3-ea7b-4426-9184-0af9d8160bb0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:12:58.889298 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682378f3-ea7b-4426-9184-0af9d8160bb0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:12:58.889575 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-682378f3-ea7b-4426-9184-0af9d8160bb0 admin admin] PATH_INFO: `/v3/projects/416c136037734f6f938896f1e48cef3a` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:12:58.891123 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682378f3-ea7b-4426-9184-0af9d8160bb0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=416c136037734f6f938896f1e48cef3a)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:12:58.893142 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-682378f3-ea7b-4426-9184-0af9d8160bb0 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:12:58.907561 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 902/1803] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jul 13 22:12:58 2022] POST /identity/v3/auth/tokens => generated 3299 bytes in 163 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 22:12:58.924886 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4297ec4d-2e11-4e73-befb-16029bf17fe5 None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:12:58.927269 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-4297ec4d-2e11-4e73-befb-16029bf17fe5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:12:58.959319 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-682378f3-ea7b-4426-9184-0af9d8160bb0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '416c136037734f6f938896f1e48cef3a', 'request_id': 'req-682378f3-ea7b-4426-9184-0af9d8160bb0'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:12:58.968795 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-682378f3-ea7b-4426-9184-0af9d8160bb0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '416c136037734f6f938896f1e48cef3a', 'request_id': 'req-682378f3-ea7b-4426-9184-0af9d8160bb0'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:12:58.975672 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-682378f3-ea7b-4426-9184-0af9d8160bb0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '416c136037734f6f938896f1e48cef3a', 'request_id': 'req-682378f3-ea7b-4426-9184-0af9d8160bb0'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:12:58.989435 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:12:59.017247 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-682378f3-ea7b-4426-9184-0af9d8160bb0 admin admin] The token cache is being invalidate because project 416c136037734f6f938896f1e48cef3a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80381) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 22:12:59.018880 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-682378f3-ea7b-4426-9184-0af9d8160bb0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:12:59.021421 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-682378f3-ea7b-4426-9184-0af9d8160bb0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:12:59.025548 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 902/1804] 10.222.0.52 () {58 vars in 1232 bytes} [Wed Jul 13 22:12:58 2022] DELETE /identity/v3/projects/416c136037734f6f938896f1e48cef3a => generated 0 bytes in 250 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 22:12:59.035950 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e64c4cb-3c5f-455d-8488-e7a76131c415 None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:12:59.036450 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e64c4cb-3c5f-455d-8488-e7a76131c415 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:12:59.036669 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e64c4cb-3c5f-455d-8488-e7a76131c415 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:12:59.060707 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4297ec4d-2e11-4e73-befb-16029bf17fe5 None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:12:59.090642 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-8e64c4cb-3c5f-455d-8488-e7a76131c415 None None] MFA Rules not processed for user `cc99c93da5074cf5a48bc2aa9a92c5e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 22:12:59.095288 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-8e64c4cb-3c5f-455d-8488-e7a76131c415 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:12:59.192630 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbabb7a-2649-4b6b-9892-e2ef9716341f service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc77e1ea1ada4700b809beb1c980c5e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417e3e2f1333481795a089bef2e24bea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:12:59.193618 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbabb7a-2649-4b6b-9892-e2ef9716341f service cinder] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:12:59.193999 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbabb7a-2649-4b6b-9892-e2ef9716341f service cinder] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:12:59.194379 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bdbabb7a-2649-4b6b-9892-e2ef9716341f service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:12:59.199471 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-bdbabb7a-2649-4b6b-9892-e2ef9716341f service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:12:59.245549 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 903/1805] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jul 13 22:12:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 211 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 22:12:59.254192 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51561bb-6d7e-4747-b41e-2b8ed4b03a12 None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:12:59.255602 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51561bb-6d7e-4747-b41e-2b8ed4b03a12 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:12:59.256170 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b51561bb-6d7e-4747-b41e-2b8ed4b03a12 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:12:59.305671 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-b51561bb-6d7e-4747-b41e-2b8ed4b03a12 None None] MFA Rules not processed for user `cc99c93da5074cf5a48bc2aa9a92c5e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 22:12:59.307970 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-b51561bb-6d7e-4747-b41e-2b8ed4b03a12 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:12:59.330723 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 904/1806] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jul 13 22:12:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 22:12:59.331649 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51561bb-6d7e-4747-b41e-2b8ed4b03a12 None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:12:59.355121 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b51561bb-6d7e-4747-b41e-2b8ed4b03a12 None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:12:59.367152 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a49529e-1113-47fb-80f0-0460062410d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:12:59.369050 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a49529e-1113-47fb-80f0-0460062410d3 admin admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:12:59.370473 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdbabb7a-2649-4b6b-9892-e2ef9716341f service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:12:59.370695 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a49529e-1113-47fb-80f0-0460062410d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:12:59.371074 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a49529e-1113-47fb-80f0-0460062410d3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:12:59.371938 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a49529e-1113-47fb-80f0-0460062410d3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 22:12:59.374013 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a49529e-1113-47fb-80f0-0460062410d3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:12:59.375874 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bdbabb7a-2649-4b6b-9892-e2ef9716341f service cinder] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:12:59.378124 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a49529e-1113-47fb-80f0-0460062410d3 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:12:59.404976 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 903/1807] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jul 13 22:12:58 2022] GET /identity/v3/auth/tokens => generated 3299 bytes in 483 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 22:12:59.407884 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 905/1808] 10.222.0.52 () {58 vars in 1152 bytes} [Wed Jul 13 22:12:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 77 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 22:12:59.412312 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbabb7a-2649-4b6b-9892-e2ef9716341f service cinder] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:12:59.453571 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bdbabb7a-2649-4b6b-9892-e2ef9716341f service cinder] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:12:59.466276 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4aa58d-4626-4cea-8b98-1bf697554b50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:12:59.467905 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4aa58d-4626-4cea-8b98-1bf697554b50 admin admin] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:12:59.469264 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4aa58d-4626-4cea-8b98-1bf697554b50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:12:59.470269 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cc4aa58d-4626-4cea-8b98-1bf697554b50 admin admin] PATH_INFO: `/v3/projects` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:12:59.471670 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4aa58d-4626-4cea-8b98-1bf697554b50 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:12:59.482239 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a49529e-1113-47fb-80f0-0460062410d3 admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:12:59.488218 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-6a49529e-1113-47fb-80f0-0460062410d3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:12:59.504651 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cc4aa58d-4626-4cea-8b98-1bf697554b50 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:12:59.515772 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a49529e-1113-47fb-80f0-0460062410d3 admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:12:59.527741 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9115a832-01ea-4e5e-a044-43b1fbc3f0e1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc77e1ea1ada4700b809beb1c980c5e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417e3e2f1333481795a089bef2e24bea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:12:59.529521 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9115a832-01ea-4e5e-a044-43b1fbc3f0e1 service cinder] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:12:59.529922 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9115a832-01ea-4e5e-a044-43b1fbc3f0e1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:12:59.530313 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9115a832-01ea-4e5e-a044-43b1fbc3f0e1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:12:59.534842 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-9115a832-01ea-4e5e-a044-43b1fbc3f0e1 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:12:59.577759 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 904/1809] 10.222.0.52 () {60 vars in 1152 bytes} [Wed Jul 13 22:12:59 2022] POST /identity/v3/projects => generated 371 bytes in 166 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 22:12:59.581676 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4aa58d-4626-4cea-8b98-1bf697554b50 admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:12:59.688371 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9115a832-01ea-4e5e-a044-43b1fbc3f0e1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:12:59.691022 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9115a832-01ea-4e5e-a044-43b1fbc3f0e1 service cinder] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:12:59.707843 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 906/1810] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jul 13 22:12:59 2022] GET /identity/v3/auth/tokens => generated 3236 bytes in 226 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 22:12:59.709570 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8559376e-92a9-43cc-be33-521ee6ea9a82 None None] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:12:59.710013 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8559376e-92a9-43cc-be33-521ee6ea9a82 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:12:59.710449 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8559376e-92a9-43cc-be33-521ee6ea9a82 None None] PATH_INFO: `/` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:12:59.712973 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 907/1811] 10.222.0.52 () {54 vars in 921 bytes} [Wed Jul 13 22:12:59 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 22:12:59.734902 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cc4aa58d-4626-4cea-8b98-1bf697554b50 admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:12:59.739310 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719d7d17-5e27-482a-95d3-124c490c660b None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:12:59.739621 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719d7d17-5e27-482a-95d3-124c490c660b None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:12:59.739892 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-719d7d17-5e27-482a-95d3-124c490c660b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:12:59.747616 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:12:59.751567 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11368d1d-3197-43f2-8d96-50b3155d7fa9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:12:59.752407 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11368d1d-3197-43f2-8d96-50b3155d7fa9 admin admin] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:12:59.754080 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11368d1d-3197-43f2-8d96-50b3155d7fa9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:12:59.755248 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11368d1d-3197-43f2-8d96-50b3155d7fa9 admin admin] PATH_INFO: `/v3/users` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:12:59.756992 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11368d1d-3197-43f2-8d96-50b3155d7fa9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:12:59.762201 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11368d1d-3197-43f2-8d96-50b3155d7fa9 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:12:59.771666 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-719d7d17-5e27-482a-95d3-124c490c660b None None] MFA Rules not processed for user `f84281091687464990ffbca590b9343b`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 22:12:59.776805 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-719d7d17-5e27-482a-95d3-124c490c660b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:12:59.827278 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 905/1812] 10.222.0.52 () {60 vars in 1143 bytes} [Wed Jul 13 22:12:59 2022] POST /identity/v3/users => generated 286 bytes in 246 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 13 22:12:59.828910 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11368d1d-3197-43f2-8d96-50b3155d7fa9 admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:12:59.846310 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11368d1d-3197-43f2-8d96-50b3155d7fa9 admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:12:59.860515 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6cc76b0-8d95-4e6b-9150-e86ac349c643 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:12:59.861210 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6cc76b0-8d95-4e6b-9150-e86ac349c643 admin admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:12:59.861494 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6cc76b0-8d95-4e6b-9150-e86ac349c643 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:12:59.861771 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c6cc76b0-8d95-4e6b-9150-e86ac349c643 admin admin] PATH_INFO: `/v3/roles` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:12:59.862571 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6cc76b0-8d95-4e6b-9150-e86ac349c643 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:12:59.865878 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c6cc76b0-8d95-4e6b-9150-e86ac349c643 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:12:59.877432 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:12:59.893502 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 906/1813] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jul 13 22:12:59 2022] GET /identity/v3/roles => generated 1420 bytes in 66 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 22:12:59.898552 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6cc76b0-8d95-4e6b-9150-e86ac349c643 admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:12:59.926573 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 908/1814] 10.222.0.52 () {58 vars in 1018 bytes} [Wed Jul 13 22:12:59 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 181 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 13 22:12:59.929135 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c6cc76b0-8d95-4e6b-9150-e86ac349c643 admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:12:59.936478 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-711f6940-b192-47a4-9ec7-e6f59720b55c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:12:59.937812 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711f6940-b192-47a4-9ec7-e6f59720b55c admin admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:12:59.937812 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711f6940-b192-47a4-9ec7-e6f59720b55c admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:12:59.938031 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-711f6940-b192-47a4-9ec7-e6f59720b55c admin admin] PATH_INFO: `/v3/roles` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:12:59.938757 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711f6940-b192-47a4-9ec7-e6f59720b55c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:12:59.941178 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-711f6940-b192-47a4-9ec7-e6f59720b55c admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:12:59.958001 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 907/1815] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jul 13 22:12:59 2022] GET /identity/v3/roles => generated 1420 bytes in 58 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 22:12:59.959893 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719d7d17-5e27-482a-95d3-124c490c660b None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:12:59.981968 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-719d7d17-5e27-482a-95d3-124c490c660b None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:12:59.989560 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fcbd209-1021-4579-8d29-603e8dc2c8d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:12:59.990883 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcbd209-1021-4579-8d29-603e8dc2c8d4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:12:59.991261 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcbd209-1021-4579-8d29-603e8dc2c8d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:12:59.991831 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8fcbd209-1021-4579-8d29-603e8dc2c8d4 admin admin] PATH_INFO: `/v3/projects/519ca221e7e7421581283c9d55a8c464/users/b710832fe67b4d7fb2ca9972372a282a/roles/2a5e86976394498abb6310697cdb3f8b` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:13:00.012610 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fcbd209-1021-4579-8d29-603e8dc2c8d4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=519ca221e7e7421581283c9d55a8c464, user_id=b710832fe67b4d7fb2ca9972372a282a, role_id=2a5e86976394498abb6310697cdb3f8b)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:13:00.016189 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8fcbd209-1021-4579-8d29-603e8dc2c8d4 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:13:00.042078 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 909/1816] 10.222.0.52 () {60 vars in 1482 bytes} [Wed Jul 13 22:12:59 2022] PUT /identity/v3/projects/519ca221e7e7421581283c9d55a8c464/users/b710832fe67b4d7fb2ca9972372a282a/roles/2a5e86976394498abb6310697cdb3f8b => generated 0 bytes in 83 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 22:13:00.052864 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b34278-a9cb-4bed-b513-0b4e235717f7 None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:13:00.053384 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b34278-a9cb-4bed-b513-0b4e235717f7 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:13:00.053740 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b34278-a9cb-4bed-b513-0b4e235717f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:13:00.091755 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-f5b34278-a9cb-4bed-b513-0b4e235717f7 None None] MFA Rules not processed for user `b710832fe67b4d7fb2ca9972372a282a`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 22:13:00.092972 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-f5b34278-a9cb-4bed-b513-0b4e235717f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:13:00.234541 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 908/1817] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jul 13 22:13:00 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 180 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 22:13:00.245170 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fcbd209-1021-4579-8d29-603e8dc2c8d4 admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:13:00.247141 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-8fcbd209-1021-4579-8d29-603e8dc2c8d4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:13:00.347247 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8fcbd209-1021-4579-8d29-603e8dc2c8d4 admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:13:00.360116 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0aaf8f-f62a-4a5c-b363-c11503ede3cc service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4d35d50e15d43d3afc59206b8165f30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417e3e2f1333481795a089bef2e24bea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:13:00.360621 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b34278-a9cb-4bed-b513-0b4e235717f7 None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:13:00.362025 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-f5b34278-a9cb-4bed-b513-0b4e235717f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:13:00.362442 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0aaf8f-f62a-4a5c-b363-c11503ede3cc service glance] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:13:00.362952 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0aaf8f-f62a-4a5c-b363-c11503ede3cc service glance] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:13:00.363295 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd0aaf8f-f62a-4a5c-b363-c11503ede3cc service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:13:00.376766 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0aaf8f-f62a-4a5c-b363-c11503ede3cc service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:13:00.380049 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd0aaf8f-f62a-4a5c-b363-c11503ede3cc service glance] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:13:00.390634 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 910/1818] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jul 13 22:13:00 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 22:13:00.496077 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b34278-a9cb-4bed-b513-0b4e235717f7 None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:13:00.501692 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c133626-f2c5-4587-aa0f-d6cb299208b2 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84281091687464990ffbca590b9343b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417e3e2f1333481795a089bef2e24bea', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:13:00.502960 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c133626-f2c5-4587-aa0f-d6cb299208b2 service nova] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:13:00.503328 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c133626-f2c5-4587-aa0f-d6cb299208b2 service nova] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:13:00.503731 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c133626-f2c5-4587-aa0f-d6cb299208b2 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:13:00.513743 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c133626-f2c5-4587-aa0f-d6cb299208b2 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:13:00.515480 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c133626-f2c5-4587-aa0f-d6cb299208b2 service nova] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:13:00.527090 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 909/1819] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jul 13 22:13:00 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 167 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 13 22:13:03.521676 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0aaf8f-f62a-4a5c-b363-c11503ede3cc service glance] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:13:03.523773 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-bd0aaf8f-f62a-4a5c-b363-c11503ede3cc service glance] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:13:03.620124 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:13:03.626502 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd0aaf8f-f62a-4a5c-b363-c11503ede3cc service glance] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:13:03.632784 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6e2f37-cc9d-4856-b66e-18657fd38960 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:13:03.634707 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6e2f37-cc9d-4856-b66e-18657fd38960 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:13:03.636071 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6e2f37-cc9d-4856-b66e-18657fd38960 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:13:03.636457 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4e6e2f37-cc9d-4856-b66e-18657fd38960 admin admin] PATH_INFO: `/v3/users/91d8566c5b0940b9ab7d0efea689839e` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:13:03.637842 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e6e2f37-cc9d-4856-b66e-18657fd38960 admin admin] RBAC: Authorizing `identity:delete_user(user_id=91d8566c5b0940b9ab7d0efea689839e)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:13:03.639949 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4e6e2f37-cc9d-4856-b66e-18657fd38960 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:13:03.701647 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-4e6e2f37-cc9d-4856-b66e-18657fd38960 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '91d8566c5b0940b9ab7d0efea689839e'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:13:03.715631 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-4e6e2f37-cc9d-4856-b66e-18657fd38960 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '91d8566c5b0940b9ab7d0efea689839e'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:13:03.732503 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-4e6e2f37-cc9d-4856-b66e-18657fd38960 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '91d8566c5b0940b9ab7d0efea689839e'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:13:03.733542 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-4e6e2f37-cc9d-4856-b66e-18657fd38960 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '91d8566c5b0940b9ab7d0efea689839e'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:13:03.734301 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-4e6e2f37-cc9d-4856-b66e-18657fd38960 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '91d8566c5b0940b9ab7d0efea689839e'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:13:03.749592 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 911/1820] 10.222.0.52 () {58 vars in 1223 bytes} [Wed Jul 13 22:13:03 2022] DELETE /identity/v3/users/91d8566c5b0940b9ab7d0efea689839e => generated 0 bytes in 228 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 22:13:03.760901 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46119aa-3bba-4501-b401-bb76cf85918b None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:13:03.761589 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46119aa-3bba-4501-b401-bb76cf85918b None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:13:03.761589 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b46119aa-3bba-4501-b401-bb76cf85918b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:13:03.782805 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-b46119aa-3bba-4501-b401-bb76cf85918b None None] MFA Rules not processed for user `cc99c93da5074cf5a48bc2aa9a92c5e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 22:13:03.784032 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-b46119aa-3bba-4501-b401-bb76cf85918b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:13:03.864030 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 910/1821] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jul 13 22:13:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 22:13:03.876320 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6e2f37-cc9d-4856-b66e-18657fd38960 admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:13:03.877801 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-4e6e2f37-cc9d-4856-b66e-18657fd38960 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:13:03.994528 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4e6e2f37-cc9d-4856-b66e-18657fd38960 admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:13:03.998743 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ea27be-58aa-4d65-9900-88f1c50c2897 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '847ec9a1bd4d4dec804318a5c845895a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417e3e2f1333481795a089bef2e24bea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:13:03.999521 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ea27be-58aa-4d65-9900-88f1c50c2897 service neutron] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:13:03.999902 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ea27be-58aa-4d65-9900-88f1c50c2897 service neutron] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:13:04.000276 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67ea27be-58aa-4d65-9900-88f1c50c2897 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:13:04.011641 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ea27be-58aa-4d65-9900-88f1c50c2897 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:13:04.013268 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67ea27be-58aa-4d65-9900-88f1c50c2897 service neutron] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:13:04.034146 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 912/1822] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jul 13 22:13:03 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 159 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 22:13:04.421229 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46119aa-3bba-4501-b401-bb76cf85918b None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:13:04.422185 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-b46119aa-3bba-4501-b401-bb76cf85918b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:13:04.433424 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:13:04.451661 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b46119aa-3bba-4501-b401-bb76cf85918b None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:13:04.464402 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e564a5-213f-47b9-bfc0-003321aa15dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:13:04.465295 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e564a5-213f-47b9-bfc0-003321aa15dd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:13:04.465604 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e564a5-213f-47b9-bfc0-003321aa15dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:13:04.465901 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a5e564a5-213f-47b9-bfc0-003321aa15dd admin admin] PATH_INFO: `/v3/projects/4ca8f8a653e44e50a620ba8b9401b574` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:13:04.467220 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e564a5-213f-47b9-bfc0-003321aa15dd admin admin] RBAC: Authorizing `identity:delete_project(project_id=4ca8f8a653e44e50a620ba8b9401b574)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:13:04.469333 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a5e564a5-213f-47b9-bfc0-003321aa15dd admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:13:04.520226 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-a5e564a5-213f-47b9-bfc0-003321aa15dd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4ca8f8a653e44e50a620ba8b9401b574', 'request_id': 'req-a5e564a5-213f-47b9-bfc0-003321aa15dd'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:13:04.528636 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-a5e564a5-213f-47b9-bfc0-003321aa15dd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4ca8f8a653e44e50a620ba8b9401b574', 'request_id': 'req-a5e564a5-213f-47b9-bfc0-003321aa15dd'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:13:04.540035 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-a5e564a5-213f-47b9-bfc0-003321aa15dd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4ca8f8a653e44e50a620ba8b9401b574', 'request_id': 'req-a5e564a5-213f-47b9-bfc0-003321aa15dd'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:13:04.559799 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-a5e564a5-213f-47b9-bfc0-003321aa15dd admin admin] The token cache is being invalidate because project 4ca8f8a653e44e50a620ba8b9401b574 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80382) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 22:13:04.560075 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-a5e564a5-213f-47b9-bfc0-003321aa15dd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:13:04.560739 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-a5e564a5-213f-47b9-bfc0-003321aa15dd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:13:04.562474 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 911/1823] 10.222.0.52 () {58 vars in 1232 bytes} [Wed Jul 13 22:13:04 2022] DELETE /identity/v3/projects/4ca8f8a653e44e50a620ba8b9401b574 => generated 0 bytes in 142 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 22:13:04.566811 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ea27be-58aa-4d65-9900-88f1c50c2897 service neutron] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:13:04.580512 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67ea27be-58aa-4d65-9900-88f1c50c2897 service neutron] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:13:04.667877 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed297e4f-82b7-43a0-9d1c-251647e8c7d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:13:04.668958 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed297e4f-82b7-43a0-9d1c-251647e8c7d3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:13:04.669336 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed297e4f-82b7-43a0-9d1c-251647e8c7d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:13:04.669910 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed297e4f-82b7-43a0-9d1c-251647e8c7d3 admin admin] PATH_INFO: `/v3/users/7a0b3364444544558aa9f88b32883710` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:13:04.672051 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed297e4f-82b7-43a0-9d1c-251647e8c7d3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7a0b3364444544558aa9f88b32883710)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:13:04.674134 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed297e4f-82b7-43a0-9d1c-251647e8c7d3 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:13:04.716521 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:13:04.771454 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-ed297e4f-82b7-43a0-9d1c-251647e8c7d3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7a0b3364444544558aa9f88b32883710'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:13:04.785707 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-ed297e4f-82b7-43a0-9d1c-251647e8c7d3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7a0b3364444544558aa9f88b32883710'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:13:04.811940 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-ed297e4f-82b7-43a0-9d1c-251647e8c7d3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7a0b3364444544558aa9f88b32883710'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:13:04.812704 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-ed297e4f-82b7-43a0-9d1c-251647e8c7d3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7a0b3364444544558aa9f88b32883710'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:13:04.813321 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-ed297e4f-82b7-43a0-9d1c-251647e8c7d3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7a0b3364444544558aa9f88b32883710'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:13:04.838733 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 913/1824] 10.222.0.52 () {58 vars in 1223 bytes} [Wed Jul 13 22:13:04 2022] DELETE /identity/v3/users/7a0b3364444544558aa9f88b32883710 => generated 0 bytes in 268 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 22:13:05.291157 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e564a5-213f-47b9-bfc0-003321aa15dd admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:13:05.293413 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-a5e564a5-213f-47b9-bfc0-003321aa15dd admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:13:05.317114 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a5e564a5-213f-47b9-bfc0-003321aa15dd admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:13:05.412532 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-030630e0-3924-4971-8716-286bb53337ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:13:05.413635 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030630e0-3924-4971-8716-286bb53337ec admin admin] REQUEST_METHOD: `DELETE` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:13:05.414005 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030630e0-3924-4971-8716-286bb53337ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:13:05.414381 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-030630e0-3924-4971-8716-286bb53337ec admin admin] PATH_INFO: `/v3/projects/056dcb29d90146638679553ed5150214` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:13:05.415775 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030630e0-3924-4971-8716-286bb53337ec admin admin] RBAC: Authorizing `identity:delete_project(project_id=056dcb29d90146638679553ed5150214)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:13:05.417772 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-030630e0-3924-4971-8716-286bb53337ec admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:13:05.502344 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-030630e0-3924-4971-8716-286bb53337ec admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '056dcb29d90146638679553ed5150214', 'request_id': 'req-030630e0-3924-4971-8716-286bb53337ec'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:13:05.512394 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-030630e0-3924-4971-8716-286bb53337ec admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '056dcb29d90146638679553ed5150214', 'request_id': 'req-030630e0-3924-4971-8716-286bb53337ec'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:13:05.519973 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-030630e0-3924-4971-8716-286bb53337ec admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '056dcb29d90146638679553ed5150214', 'request_id': 'req-030630e0-3924-4971-8716-286bb53337ec'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:13:05.549488 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-030630e0-3924-4971-8716-286bb53337ec admin admin] The token cache is being invalidate because project 056dcb29d90146638679553ed5150214 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80382) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 22:13:05.550151 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-030630e0-3924-4971-8716-286bb53337ec admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:13:05.551936 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-030630e0-3924-4971-8716-286bb53337ec admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:13:05.555148 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 912/1825] 10.222.0.52 () {58 vars in 1232 bytes} [Wed Jul 13 22:13:05 2022] DELETE /identity/v3/projects/056dcb29d90146638679553ed5150214 => generated 0 bytes in 269 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 22:13:05.560960 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1167066d-f43f-454b-abea-a8163c9ae599 None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:13:05.562085 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1167066d-f43f-454b-abea-a8163c9ae599 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:13:05.562690 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1167066d-f43f-454b-abea-a8163c9ae599 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:13:05.589558 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-1167066d-f43f-454b-abea-a8163c9ae599 None None] MFA Rules not processed for user `cc99c93da5074cf5a48bc2aa9a92c5e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 22:13:05.593682 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-1167066d-f43f-454b-abea-a8163c9ae599 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:13:05.692693 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 914/1826] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jul 13 22:13:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 133 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 13 22:13:05.700876 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fbfd52-73df-43ab-bc7d-4ea2d023457c None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:13:05.701430 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fbfd52-73df-43ab-bc7d-4ea2d023457c None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:13:05.701830 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12fbfd52-73df-43ab-bc7d-4ea2d023457c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:13:05.751434 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-12fbfd52-73df-43ab-bc7d-4ea2d023457c None None] MFA Rules not processed for user `cc99c93da5074cf5a48bc2aa9a92c5e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 22:13:05.752942 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-12fbfd52-73df-43ab-bc7d-4ea2d023457c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:13:05.772305 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 913/1827] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jul 13 22:13:05 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 72 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 22:13:05.777678 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1167066d-f43f-454b-abea-a8163c9ae599 None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:13:05.804170 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1167066d-f43f-454b-abea-a8163c9ae599 None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:13:05.811180 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130b70e0-4ad3-43e1-b53a-d4111dfc64fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:13:05.812057 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130b70e0-4ad3-43e1-b53a-d4111dfc64fb admin admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:13:05.812340 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130b70e0-4ad3-43e1-b53a-d4111dfc64fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:13:05.812704 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-130b70e0-4ad3-43e1-b53a-d4111dfc64fb admin admin] PATH_INFO: `/v3/domains` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:13:05.814227 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130b70e0-4ad3-43e1-b53a-d4111dfc64fb admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 22:13:05.814562 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130b70e0-4ad3-43e1-b53a-d4111dfc64fb admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:13:05.816231 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-130b70e0-4ad3-43e1-b53a-d4111dfc64fb admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:13:05.855031 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 915/1828] 10.222.0.52 () {58 vars in 1152 bytes} [Wed Jul 13 22:13:05 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 79 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 22:13:05.859730 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12fbfd52-73df-43ab-bc7d-4ea2d023457c None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:13:05.871087 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12fbfd52-73df-43ab-bc7d-4ea2d023457c None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:13:05.878223 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bbbb7d4-cb57-4da3-9ab0-d84f54e9feca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:13:05.880011 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbbb7d4-cb57-4da3-9ab0-d84f54e9feca admin admin] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:13:05.880472 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbbb7d4-cb57-4da3-9ab0-d84f54e9feca admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:13:05.880926 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3bbbb7d4-cb57-4da3-9ab0-d84f54e9feca admin admin] PATH_INFO: `/v3/projects` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:13:05.881793 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bbbb7d4-cb57-4da3-9ab0-d84f54e9feca admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:13:05.884183 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3bbbb7d4-cb57-4da3-9ab0-d84f54e9feca admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:13:05.894302 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:13:05.928605 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 914/1829] 10.222.0.52 () {60 vars in 1152 bytes} [Wed Jul 13 22:13:05 2022] POST /identity/v3/projects => generated 381 bytes in 67 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 22:13:05.930093 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130b70e0-4ad3-43e1-b53a-d4111dfc64fb admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:13:05.943772 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-130b70e0-4ad3-43e1-b53a-d4111dfc64fb admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:13:06.015252 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a74637-a6ad-4378-bd67-98beff352ee7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:13:06.016051 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a74637-a6ad-4378-bd67-98beff352ee7 admin admin] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:13:06.016424 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a74637-a6ad-4378-bd67-98beff352ee7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:13:06.017261 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77a74637-a6ad-4378-bd67-98beff352ee7 admin admin] PATH_INFO: `/v3/users` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:13:06.018094 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a74637-a6ad-4378-bd67-98beff352ee7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:13:06.020655 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-77a74637-a6ad-4378-bd67-98beff352ee7 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:13:06.052826 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 916/1830] 10.222.0.52 () {60 vars in 1143 bytes} [Wed Jul 13 22:13:05 2022] POST /identity/v3/users => generated 291 bytes in 120 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 22:13:06.054739 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bbbb7d4-cb57-4da3-9ab0-d84f54e9feca admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:13:06.065148 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3bbbb7d4-cb57-4da3-9ab0-d84f54e9feca admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:13:06.069926 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40b21f8-1a72-4401-ba80-6c6bda4ecdb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:13:06.070763 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40b21f8-1a72-4401-ba80-6c6bda4ecdb9 admin admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:13:06.071042 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40b21f8-1a72-4401-ba80-6c6bda4ecdb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:13:06.071712 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e40b21f8-1a72-4401-ba80-6c6bda4ecdb9 admin admin] PATH_INFO: `/v3/roles` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:13:06.072048 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e40b21f8-1a72-4401-ba80-6c6bda4ecdb9 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:13:06.074170 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e40b21f8-1a72-4401-ba80-6c6bda4ecdb9 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:13:06.097363 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 915/1831] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jul 13 22:13:06 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 22:13:06.099895 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a74637-a6ad-4378-bd67-98beff352ee7 admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:13:06.113908 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77a74637-a6ad-4378-bd67-98beff352ee7 admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:13:06.121967 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-acb606fb-c3ca-4829-a22d-c9a4db4ffdc3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:13:06.121967 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb606fb-c3ca-4829-a22d-c9a4db4ffdc3 admin admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:13:06.121967 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb606fb-c3ca-4829-a22d-c9a4db4ffdc3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:13:06.122375 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acb606fb-c3ca-4829-a22d-c9a4db4ffdc3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:13:06.123183 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb606fb-c3ca-4829-a22d-c9a4db4ffdc3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:13:06.124818 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-acb606fb-c3ca-4829-a22d-c9a4db4ffdc3 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:13:06.142279 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 917/1832] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jul 13 22:13:06 2022] GET /identity/v3/roles => generated 1420 bytes in 43 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 22:13:06.143151 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40b21f8-1a72-4401-ba80-6c6bda4ecdb9 admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:13:06.154718 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e40b21f8-1a72-4401-ba80-6c6bda4ecdb9 admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:13:06.159912 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5381f61-9fe3-4428-a5ac-1bdcd38a6f92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:13:06.161216 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5381f61-9fe3-4428-a5ac-1bdcd38a6f92 admin admin] REQUEST_METHOD: `PUT` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:13:06.161216 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5381f61-9fe3-4428-a5ac-1bdcd38a6f92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:13:06.161684 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b5381f61-9fe3-4428-a5ac-1bdcd38a6f92 admin admin] PATH_INFO: `/v3/projects/37ae216ef2924deea089cfc1baa1d999/users/3c5360f8badf4cf991343d73643d52d9/roles/2a5e86976394498abb6310697cdb3f8b` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:13:06.180404 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5381f61-9fe3-4428-a5ac-1bdcd38a6f92 admin admin] RBAC: Authorizing `identity:create_grant(project_id=37ae216ef2924deea089cfc1baa1d999, user_id=3c5360f8badf4cf991343d73643d52d9, role_id=2a5e86976394498abb6310697cdb3f8b)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:13:06.184202 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b5381f61-9fe3-4428-a5ac-1bdcd38a6f92 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:13:06.206411 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 916/1833] 10.222.0.52 () {60 vars in 1482 bytes} [Wed Jul 13 22:13:06 2022] PUT /identity/v3/projects/37ae216ef2924deea089cfc1baa1d999/users/3c5360f8badf4cf991343d73643d52d9/roles/2a5e86976394498abb6310697cdb3f8b => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 22:13:06.219168 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be95e1a3-e0ba-433c-9b26-b01c22504ac1 None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:13:06.219531 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be95e1a3-e0ba-433c-9b26-b01c22504ac1 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:13:06.220179 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be95e1a3-e0ba-433c-9b26-b01c22504ac1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:13:06.226758 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:DELETE=8,keystone:SELECT=88,keystone:INSERT=4 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:13:06.258361 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-be95e1a3-e0ba-433c-9b26-b01c22504ac1 None None] MFA Rules not processed for user `3c5360f8badf4cf991343d73643d52d9`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 22:13:06.260383 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-be95e1a3-e0ba-433c-9b26-b01c22504ac1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:13:06.328659 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 918/1834] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jul 13 22:13:06 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 22:13:06.333334 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5381f61-9fe3-4428-a5ac-1bdcd38a6f92 admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:13:06.415444 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b5381f61-9fe3-4428-a5ac-1bdcd38a6f92 admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:13:06.420538 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c895f9-9a64-46e7-9e12-1a522075f758 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:13:06.421208 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c895f9-9a64-46e7-9e12-1a522075f758 admin admin] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:13:06.421483 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c895f9-9a64-46e7-9e12-1a522075f758 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:13:06.421751 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e7c895f9-9a64-46e7-9e12-1a522075f758 admin admin] PATH_INFO: `/v3/projects` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:13:06.422466 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c895f9-9a64-46e7-9e12-1a522075f758 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:13:06.424413 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e7c895f9-9a64-46e7-9e12-1a522075f758 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:13:06.469117 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 917/1835] 10.222.0.52 () {60 vars in 1152 bytes} [Wed Jul 13 22:13:06 2022] POST /identity/v3/projects => generated 381 bytes in 136 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 22:13:06.473209 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be95e1a3-e0ba-433c-9b26-b01c22504ac1 None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:13:06.491624 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be95e1a3-e0ba-433c-9b26-b01c22504ac1 None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:13:06.550598 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31758825-6dca-4d01-984b-afd48d8a24f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:13:06.551176 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31758825-6dca-4d01-984b-afd48d8a24f7 admin admin] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:13:06.551460 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31758825-6dca-4d01-984b-afd48d8a24f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:13:06.551769 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31758825-6dca-4d01-984b-afd48d8a24f7 admin admin] PATH_INFO: `/v3/users` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:13:06.552488 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31758825-6dca-4d01-984b-afd48d8a24f7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:13:06.554516 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-31758825-6dca-4d01-984b-afd48d8a24f7 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:13:06.577681 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 919/1836] 10.222.0.52 () {60 vars in 1143 bytes} [Wed Jul 13 22:13:06 2022] POST /identity/v3/users => generated 297 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 22:13:06.581463 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c895f9-9a64-46e7-9e12-1a522075f758 admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:13:06.593289 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e7c895f9-9a64-46e7-9e12-1a522075f758 admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:13:06.598203 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b17c44-268d-46f8-aab2-9a12c79bf4cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:13:06.598966 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b17c44-268d-46f8-aab2-9a12c79bf4cf admin admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:13:06.599247 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b17c44-268d-46f8-aab2-9a12c79bf4cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:13:06.599738 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6b17c44-268d-46f8-aab2-9a12c79bf4cf admin admin] PATH_INFO: `/v3/roles` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:13:06.600616 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b17c44-268d-46f8-aab2-9a12c79bf4cf admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:13:06.608243 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6b17c44-268d-46f8-aab2-9a12c79bf4cf admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:13:06.618986 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 918/1837] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jul 13 22:13:06 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 22:13:06.623756 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31758825-6dca-4d01-984b-afd48d8a24f7 admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:13:06.634832 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-31758825-6dca-4d01-984b-afd48d8a24f7 admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:13:06.640322 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079ee9e3-f6be-4b16-bbe4-33d01f8d634a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:13:06.641334 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079ee9e3-f6be-4b16-bbe4-33d01f8d634a admin admin] REQUEST_METHOD: `PUT` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:13:06.641712 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079ee9e3-f6be-4b16-bbe4-33d01f8d634a admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:13:06.642089 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-079ee9e3-f6be-4b16-bbe4-33d01f8d634a admin admin] PATH_INFO: `/v3/projects/7973f64abb8e42c18f84feab559decf2/users/0544f477b5ca4710949d1aab30da606a/roles/44efba2579f342df8a0789c59d163ec3` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:13:06.658732 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-079ee9e3-f6be-4b16-bbe4-33d01f8d634a admin admin] RBAC: Authorizing `identity:create_grant(project_id=7973f64abb8e42c18f84feab559decf2, user_id=0544f477b5ca4710949d1aab30da606a, role_id=44efba2579f342df8a0789c59d163ec3)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:13:06.661400 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-079ee9e3-f6be-4b16-bbe4-33d01f8d634a admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:13:06.677423 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 920/1838] 10.222.0.52 () {60 vars in 1482 bytes} [Wed Jul 13 22:13:06 2022] PUT /identity/v3/projects/7973f64abb8e42c18f84feab559decf2/users/0544f477b5ca4710949d1aab30da606a/roles/44efba2579f342df8a0789c59d163ec3 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 22:13:06.681734 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b17c44-268d-46f8-aab2-9a12c79bf4cf admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:13:06.692385 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6b17c44-268d-46f8-aab2-9a12c79bf4cf admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:13:06.765583 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6ecbacda-91fc-4296-b7b5-07264d7a8146 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:13:06.766655 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecbacda-91fc-4296-b7b5-07264d7a8146 admin admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:13:06.767264 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecbacda-91fc-4296-b7b5-07264d7a8146 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:13:06.767494 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6ecbacda-91fc-4296-b7b5-07264d7a8146 admin admin] PATH_INFO: `/v3/roles` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:13:06.771137 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ecbacda-91fc-4296-b7b5-07264d7a8146 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:13:06.774749 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6ecbacda-91fc-4296-b7b5-07264d7a8146 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:13:06.791953 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 919/1839] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jul 13 22:13:06 2022] GET /identity/v3/roles => generated 1420 bytes in 111 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 22:13:06.796572 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079ee9e3-f6be-4b16-bbe4-33d01f8d634a admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:13:06.807683 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-079ee9e3-f6be-4b16-bbe4-33d01f8d634a admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:13:06.818537 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb35ce5-7309-4237-bacc-e399a44dbc0e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:13:06.819557 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb35ce5-7309-4237-bacc-e399a44dbc0e admin admin] REQUEST_METHOD: `PUT` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:13:06.819952 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb35ce5-7309-4237-bacc-e399a44dbc0e admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:13:06.820234 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3fb35ce5-7309-4237-bacc-e399a44dbc0e admin admin] PATH_INFO: `/v3/projects/7973f64abb8e42c18f84feab559decf2/users/0544f477b5ca4710949d1aab30da606a/roles/44efba2579f342df8a0789c59d163ec3` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:13:06.823282 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb35ce5-7309-4237-bacc-e399a44dbc0e admin admin] RBAC: Authorizing `identity:create_grant(project_id=7973f64abb8e42c18f84feab559decf2, user_id=0544f477b5ca4710949d1aab30da606a, role_id=44efba2579f342df8a0789c59d163ec3)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:13:06.827288 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3fb35ce5-7309-4237-bacc-e399a44dbc0e admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:13:06.836882 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 921/1840] 10.222.0.52 () {60 vars in 1482 bytes} [Wed Jul 13 22:13:06 2022] PUT /identity/v3/projects/7973f64abb8e42c18f84feab559decf2/users/0544f477b5ca4710949d1aab30da606a/roles/44efba2579f342df8a0789c59d163ec3 => generated 0 bytes in 41 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 22:13:06.847387 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5269fee-3b8f-435c-aef7-48a1a4c20ce8 None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:13:06.847675 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5269fee-3b8f-435c-aef7-48a1a4c20ce8 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:13:06.847993 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c5269fee-3b8f-435c-aef7-48a1a4c20ce8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:13:06.875441 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-c5269fee-3b8f-435c-aef7-48a1a4c20ce8 None None] MFA Rules not processed for user `0544f477b5ca4710949d1aab30da606a`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 22:13:06.876663 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-c5269fee-3b8f-435c-aef7-48a1a4c20ce8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:13:06.892538 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:13:06.961958 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 920/1841] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jul 13 22:13:06 2022] POST /identity/v3/auth/tokens => generated 3315 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 22:13:06.973093 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb35ce5-7309-4237-bacc-e399a44dbc0e admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:13:06.975107 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-3fb35ce5-7309-4237-bacc-e399a44dbc0e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:13:07.031402 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:13:07.043454 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3fb35ce5-7309-4237-bacc-e399a44dbc0e admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:13:07.047296 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-299b4734-0da1-428a-917b-f0f6f69318a0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc77e1ea1ada4700b809beb1c980c5e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417e3e2f1333481795a089bef2e24bea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:13:07.047946 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299b4734-0da1-428a-917b-f0f6f69318a0 service cinder] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:13:07.048225 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299b4734-0da1-428a-917b-f0f6f69318a0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:13:07.048496 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-299b4734-0da1-428a-917b-f0f6f69318a0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:13:07.057634 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299b4734-0da1-428a-917b-f0f6f69318a0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:13:07.059163 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-299b4734-0da1-428a-917b-f0f6f69318a0 service cinder] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:13:07.068568 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 922/1842] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jul 13 22:13:06 2022] GET /identity/v3/auth/tokens => generated 3315 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 22:13:07.840098 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5269fee-3b8f-435c-aef7-48a1a4c20ce8 None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:13:07.951648 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c5269fee-3b8f-435c-aef7-48a1a4c20ce8 None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:13:07.959074 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38419137-cd3e-4fff-a465-93d729d624a0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:13:07.960066 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38419137-cd3e-4fff-a465-93d729d624a0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:13:07.960407 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38419137-cd3e-4fff-a465-93d729d624a0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:13:07.960715 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38419137-cd3e-4fff-a465-93d729d624a0 admin admin] PATH_INFO: `/v3/users/3c5360f8badf4cf991343d73643d52d9` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:13:07.961982 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38419137-cd3e-4fff-a465-93d729d624a0 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3c5360f8badf4cf991343d73643d52d9)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:13:07.964407 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38419137-cd3e-4fff-a465-93d729d624a0 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:13:08.039634 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-38419137-cd3e-4fff-a465-93d729d624a0 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3c5360f8badf4cf991343d73643d52d9'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:13:08.050380 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-38419137-cd3e-4fff-a465-93d729d624a0 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3c5360f8badf4cf991343d73643d52d9'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:13:08.063891 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-38419137-cd3e-4fff-a465-93d729d624a0 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3c5360f8badf4cf991343d73643d52d9'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:13:08.064541 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-38419137-cd3e-4fff-a465-93d729d624a0 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3c5360f8badf4cf991343d73643d52d9'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:13:08.065292 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-38419137-cd3e-4fff-a465-93d729d624a0 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3c5360f8badf4cf991343d73643d52d9'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:13:08.078024 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 921/1843] 10.222.0.52 () {58 vars in 1223 bytes} [Wed Jul 13 22:13:07 2022] DELETE /identity/v3/users/3c5360f8badf4cf991343d73643d52d9 => generated 0 bytes in 239 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 22:13:08.082845 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a99e954-5ea0-43b2-b218-ee5d86a03c26 None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:13:08.083328 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a99e954-5ea0-43b2-b218-ee5d86a03c26 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:13:08.083460 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8a99e954-5ea0-43b2-b218-ee5d86a03c26 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:13:08.103056 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-8a99e954-5ea0-43b2-b218-ee5d86a03c26 None None] MFA Rules not processed for user `cc99c93da5074cf5a48bc2aa9a92c5e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 22:13:08.104295 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-8a99e954-5ea0-43b2-b218-ee5d86a03c26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:13:08.167242 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 923/1844] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jul 13 22:13:08 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 22:13:08.177184 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38419137-cd3e-4fff-a465-93d729d624a0 admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:13:08.178674 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-38419137-cd3e-4fff-a465-93d729d624a0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:13:08.218825 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:13:08.249787 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38419137-cd3e-4fff-a465-93d729d624a0 admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:13:08.253535 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfaab1d-c7c8-48ed-8c55-fee2e6db1916 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '847ec9a1bd4d4dec804318a5c845895a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417e3e2f1333481795a089bef2e24bea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:13:08.254189 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfaab1d-c7c8-48ed-8c55-fee2e6db1916 service neutron] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:13:08.254464 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfaab1d-c7c8-48ed-8c55-fee2e6db1916 service neutron] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:13:08.254766 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6dfaab1d-c7c8-48ed-8c55-fee2e6db1916 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:13:08.266543 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dfaab1d-c7c8-48ed-8c55-fee2e6db1916 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:13:08.268330 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6dfaab1d-c7c8-48ed-8c55-fee2e6db1916 service neutron] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:13:08.283435 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 922/1845] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jul 13 22:13:08 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 22:13:08.630306 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a99e954-5ea0-43b2-b218-ee5d86a03c26 None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:13:08.632141 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-8a99e954-5ea0-43b2-b218-ee5d86a03c26 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:13:08.648834 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8a99e954-5ea0-43b2-b218-ee5d86a03c26 None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:13:08.655182 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e5b2e0-3a36-4575-9f9f-bc1826ac7208 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:13:08.656154 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e5b2e0-3a36-4575-9f9f-bc1826ac7208 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:13:08.656499 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e5b2e0-3a36-4575-9f9f-bc1826ac7208 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:13:08.656985 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61e5b2e0-3a36-4575-9f9f-bc1826ac7208 admin admin] PATH_INFO: `/v3/projects/37ae216ef2924deea089cfc1baa1d999` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:13:08.658206 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e5b2e0-3a36-4575-9f9f-bc1826ac7208 admin admin] RBAC: Authorizing `identity:delete_project(project_id=37ae216ef2924deea089cfc1baa1d999)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:13:08.661160 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61e5b2e0-3a36-4575-9f9f-bc1826ac7208 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:13:08.720607 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-61e5b2e0-3a36-4575-9f9f-bc1826ac7208 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '37ae216ef2924deea089cfc1baa1d999', 'request_id': 'req-61e5b2e0-3a36-4575-9f9f-bc1826ac7208'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:13:08.727070 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-61e5b2e0-3a36-4575-9f9f-bc1826ac7208 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '37ae216ef2924deea089cfc1baa1d999', 'request_id': 'req-61e5b2e0-3a36-4575-9f9f-bc1826ac7208'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:13:08.732655 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-61e5b2e0-3a36-4575-9f9f-bc1826ac7208 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '37ae216ef2924deea089cfc1baa1d999', 'request_id': 'req-61e5b2e0-3a36-4575-9f9f-bc1826ac7208'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:13:08.747244 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-61e5b2e0-3a36-4575-9f9f-bc1826ac7208 admin admin] The token cache is being invalidate because project 37ae216ef2924deea089cfc1baa1d999 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80381) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 22:13:08.747483 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-61e5b2e0-3a36-4575-9f9f-bc1826ac7208 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:13:08.748254 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-61e5b2e0-3a36-4575-9f9f-bc1826ac7208 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:13:08.750464 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 924/1846] 10.222.0.52 () {58 vars in 1232 bytes} [Wed Jul 13 22:13:08 2022] DELETE /identity/v3/projects/37ae216ef2924deea089cfc1baa1d999 => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 22:13:08.759865 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfaab1d-c7c8-48ed-8c55-fee2e6db1916 service neutron] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:13:08.770417 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6dfaab1d-c7c8-48ed-8c55-fee2e6db1916 service neutron] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:13:08.825557 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c043ee5-dded-43c2-86cf-97b9fe7d7880 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:13:08.826355 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c043ee5-dded-43c2-86cf-97b9fe7d7880 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:13:08.826670 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c043ee5-dded-43c2-86cf-97b9fe7d7880 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:13:08.826949 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c043ee5-dded-43c2-86cf-97b9fe7d7880 admin admin] PATH_INFO: `/v3/users/0544f477b5ca4710949d1aab30da606a` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:13:08.828134 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c043ee5-dded-43c2-86cf-97b9fe7d7880 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0544f477b5ca4710949d1aab30da606a)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:13:08.830058 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c043ee5-dded-43c2-86cf-97b9fe7d7880 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:13:08.894685 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-8c043ee5-dded-43c2-86cf-97b9fe7d7880 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0544f477b5ca4710949d1aab30da606a'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:13:08.906284 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-8c043ee5-dded-43c2-86cf-97b9fe7d7880 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0544f477b5ca4710949d1aab30da606a'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:13:08.921953 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-8c043ee5-dded-43c2-86cf-97b9fe7d7880 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0544f477b5ca4710949d1aab30da606a'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:13:08.922728 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-8c043ee5-dded-43c2-86cf-97b9fe7d7880 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0544f477b5ca4710949d1aab30da606a'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:13:08.923371 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-8c043ee5-dded-43c2-86cf-97b9fe7d7880 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0544f477b5ca4710949d1aab30da606a'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:13:08.941964 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 923/1847] 10.222.0.52 () {58 vars in 1223 bytes} [Wed Jul 13 22:13:08 2022] DELETE /identity/v3/users/0544f477b5ca4710949d1aab30da606a => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 22:13:09.340234 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e5b2e0-3a36-4575-9f9f-bc1826ac7208 admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:13:09.342844 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-61e5b2e0-3a36-4575-9f9f-bc1826ac7208 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:13:09.369420 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61e5b2e0-3a36-4575-9f9f-bc1826ac7208 admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:13:09.466275 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd743bb-9a20-46fa-81c8-a07ef6361f7d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:13:09.467428 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd743bb-9a20-46fa-81c8-a07ef6361f7d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:13:09.467819 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd743bb-9a20-46fa-81c8-a07ef6361f7d admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:13:09.468226 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd743bb-9a20-46fa-81c8-a07ef6361f7d admin admin] PATH_INFO: `/v3/projects/7973f64abb8e42c18f84feab559decf2` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:13:09.469604 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd743bb-9a20-46fa-81c8-a07ef6361f7d admin admin] RBAC: Authorizing `identity:delete_project(project_id=7973f64abb8e42c18f84feab559decf2)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:13:09.472112 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd743bb-9a20-46fa-81c8-a07ef6361f7d admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:13:09.487848 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:13:09.546174 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-1fd743bb-9a20-46fa-81c8-a07ef6361f7d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '7973f64abb8e42c18f84feab559decf2', 'request_id': 'req-1fd743bb-9a20-46fa-81c8-a07ef6361f7d'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:13:09.553898 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-1fd743bb-9a20-46fa-81c8-a07ef6361f7d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '7973f64abb8e42c18f84feab559decf2', 'request_id': 'req-1fd743bb-9a20-46fa-81c8-a07ef6361f7d'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:13:09.560891 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-1fd743bb-9a20-46fa-81c8-a07ef6361f7d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '7973f64abb8e42c18f84feab559decf2', 'request_id': 'req-1fd743bb-9a20-46fa-81c8-a07ef6361f7d'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:13:09.588743 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-1fd743bb-9a20-46fa-81c8-a07ef6361f7d admin admin] The token cache is being invalidate because project 7973f64abb8e42c18f84feab559decf2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80381) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 22:13:09.589996 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-1fd743bb-9a20-46fa-81c8-a07ef6361f7d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:13:09.590895 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-1fd743bb-9a20-46fa-81c8-a07ef6361f7d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:13:09.593235 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 925/1848] 10.222.0.52 () {58 vars in 1232 bytes} [Wed Jul 13 22:13:09 2022] DELETE /identity/v3/projects/7973f64abb8e42c18f84feab559decf2 => generated 0 bytes in 253 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 22:13:09.598059 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891bf1ba-b518-43dd-a6b3-ecfc95482c27 None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:13:09.598354 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891bf1ba-b518-43dd-a6b3-ecfc95482c27 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:13:09.598671 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-891bf1ba-b518-43dd-a6b3-ecfc95482c27 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:13:09.620630 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-891bf1ba-b518-43dd-a6b3-ecfc95482c27 None None] MFA Rules not processed for user `cc99c93da5074cf5a48bc2aa9a92c5e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 22:13:09.622464 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-891bf1ba-b518-43dd-a6b3-ecfc95482c27 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:13:09.702033 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:13:09.731660 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 924/1849] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jul 13 22:13:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 127 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 22:13:09.734602 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83ee140-4014-4b30-8002-85cc749bf5fb None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:13:09.735087 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83ee140-4014-4b30-8002-85cc749bf5fb None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:13:09.735942 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e83ee140-4014-4b30-8002-85cc749bf5fb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:13:09.770666 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-e83ee140-4014-4b30-8002-85cc749bf5fb None None] MFA Rules not processed for user `cc99c93da5074cf5a48bc2aa9a92c5e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 22:13:09.772314 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-e83ee140-4014-4b30-8002-85cc749bf5fb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:13:09.781337 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 926/1850] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jul 13 22:13:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 13 22:13:09.785756 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891bf1ba-b518-43dd-a6b3-ecfc95482c27 None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:13:09.796891 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-891bf1ba-b518-43dd-a6b3-ecfc95482c27 None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:13:09.801776 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43afb2f3-100c-4055-bbfa-30adfeb2e26a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:13:09.802535 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43afb2f3-100c-4055-bbfa-30adfeb2e26a admin admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:13:09.802883 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43afb2f3-100c-4055-bbfa-30adfeb2e26a admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:13:09.803218 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43afb2f3-100c-4055-bbfa-30adfeb2e26a admin admin] PATH_INFO: `/v3/domains` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:13:09.804076 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43afb2f3-100c-4055-bbfa-30adfeb2e26a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 22:13:09.804443 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43afb2f3-100c-4055-bbfa-30adfeb2e26a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:13:09.806461 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43afb2f3-100c-4055-bbfa-30adfeb2e26a admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:13:09.825249 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 925/1851] 10.222.0.52 () {58 vars in 1152 bytes} [Wed Jul 13 22:13:09 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 22:13:09.830315 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83ee140-4014-4b30-8002-85cc749bf5fb None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:13:09.840514 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e83ee140-4014-4b30-8002-85cc749bf5fb None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:13:09.845972 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db0291a-2ab3-4dd6-ab14-9878a9dcc4c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:13:09.847022 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db0291a-2ab3-4dd6-ab14-9878a9dcc4c4 admin admin] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:13:09.847474 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db0291a-2ab3-4dd6-ab14-9878a9dcc4c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:13:09.847855 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9db0291a-2ab3-4dd6-ab14-9878a9dcc4c4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:13:09.849000 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db0291a-2ab3-4dd6-ab14-9878a9dcc4c4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:13:09.851668 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9db0291a-2ab3-4dd6-ab14-9878a9dcc4c4 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:13:09.886590 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 927/1852] 10.222.0.52 () {60 vars in 1152 bytes} [Wed Jul 13 22:13:09 2022] POST /identity/v3/projects => generated 385 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 22:13:09.890778 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43afb2f3-100c-4055-bbfa-30adfeb2e26a admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:13:09.907754 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43afb2f3-100c-4055-bbfa-30adfeb2e26a admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:13:09.988518 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc38d361-e7b0-409f-928f-c2dd1625f2d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:13:09.989214 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc38d361-e7b0-409f-928f-c2dd1625f2d2 admin admin] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:13:09.989500 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc38d361-e7b0-409f-928f-c2dd1625f2d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:13:09.989775 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fc38d361-e7b0-409f-928f-c2dd1625f2d2 admin admin] PATH_INFO: `/v3/users` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:13:09.990551 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc38d361-e7b0-409f-928f-c2dd1625f2d2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:13:09.992917 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fc38d361-e7b0-409f-928f-c2dd1625f2d2 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:13:10.017084 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 926/1853] 10.222.0.52 () {60 vars in 1143 bytes} [Wed Jul 13 22:13:09 2022] POST /identity/v3/users => generated 293 bytes in 127 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 22:13:10.020963 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db0291a-2ab3-4dd6-ab14-9878a9dcc4c4 admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:13:10.032060 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9db0291a-2ab3-4dd6-ab14-9878a9dcc4c4 admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:13:10.037176 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9502e7ee-e8d6-409d-990c-f29ded2e7ab5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:13:10.037885 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9502e7ee-e8d6-409d-990c-f29ded2e7ab5 admin admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:13:10.038187 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9502e7ee-e8d6-409d-990c-f29ded2e7ab5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:13:10.038555 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9502e7ee-e8d6-409d-990c-f29ded2e7ab5 admin admin] PATH_INFO: `/v3/roles` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:13:10.039419 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9502e7ee-e8d6-409d-990c-f29ded2e7ab5 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:13:10.044309 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9502e7ee-e8d6-409d-990c-f29ded2e7ab5 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:13:10.058083 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 928/1854] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jul 13 22:13:10 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 22:13:10.063345 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc38d361-e7b0-409f-928f-c2dd1625f2d2 admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:13:10.075849 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fc38d361-e7b0-409f-928f-c2dd1625f2d2 admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:13:10.082819 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-41145d5b-c279-47f6-9f37-86ef13cf2d17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:13:10.083558 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41145d5b-c279-47f6-9f37-86ef13cf2d17 admin admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:13:10.083843 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41145d5b-c279-47f6-9f37-86ef13cf2d17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:13:10.084139 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-41145d5b-c279-47f6-9f37-86ef13cf2d17 admin admin] PATH_INFO: `/v3/roles` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:13:10.085066 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41145d5b-c279-47f6-9f37-86ef13cf2d17 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:13:10.087402 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-41145d5b-c279-47f6-9f37-86ef13cf2d17 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:13:10.105206 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 927/1855] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jul 13 22:13:10 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 22:13:10.106526 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9502e7ee-e8d6-409d-990c-f29ded2e7ab5 admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:13:10.126115 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9502e7ee-e8d6-409d-990c-f29ded2e7ab5 admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:13:10.132774 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec169d5-4121-4732-825d-848ca514d2a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:13:10.134786 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec169d5-4121-4732-825d-848ca514d2a3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:13:10.135293 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec169d5-4121-4732-825d-848ca514d2a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:13:10.137084 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9ec169d5-4121-4732-825d-848ca514d2a3 admin admin] PATH_INFO: `/v3/projects/20056966506b46709e31c2f102451d1d/users/c051bca299ec44e38c68e87674a7009d/roles/2a5e86976394498abb6310697cdb3f8b` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:13:10.156199 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec169d5-4121-4732-825d-848ca514d2a3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=20056966506b46709e31c2f102451d1d, user_id=c051bca299ec44e38c68e87674a7009d, role_id=2a5e86976394498abb6310697cdb3f8b)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:13:10.159290 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9ec169d5-4121-4732-825d-848ca514d2a3 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:13:10.193799 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 929/1856] 10.222.0.52 () {60 vars in 1482 bytes} [Wed Jul 13 22:13:10 2022] PUT /identity/v3/projects/20056966506b46709e31c2f102451d1d/users/c051bca299ec44e38c68e87674a7009d/roles/2a5e86976394498abb6310697cdb3f8b => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 22:13:10.197007 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd62b3d-81a2-4a88-af47-79c313301d94 None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:13:10.197613 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd62b3d-81a2-4a88-af47-79c313301d94 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:13:10.198167 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dcd62b3d-81a2-4a88-af47-79c313301d94 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:13:10.227903 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-dcd62b3d-81a2-4a88-af47-79c313301d94 None None] MFA Rules not processed for user `c051bca299ec44e38c68e87674a7009d`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 22:13:10.229183 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-dcd62b3d-81a2-4a88-af47-79c313301d94 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:13:10.306380 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 928/1857] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jul 13 22:13:10 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 22:13:10.324614 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec169d5-4121-4732-825d-848ca514d2a3 admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:13:10.326822 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-9ec169d5-4121-4732-825d-848ca514d2a3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:13:10.393619 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9ec169d5-4121-4732-825d-848ca514d2a3 admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:13:10.397982 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d99162dc-75f1-434f-8e7c-ae35f3fc1f70 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc77e1ea1ada4700b809beb1c980c5e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417e3e2f1333481795a089bef2e24bea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:13:10.398712 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99162dc-75f1-434f-8e7c-ae35f3fc1f70 service cinder] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:13:10.398995 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99162dc-75f1-434f-8e7c-ae35f3fc1f70 service cinder] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:13:10.399275 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d99162dc-75f1-434f-8e7c-ae35f3fc1f70 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:13:10.408370 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99162dc-75f1-434f-8e7c-ae35f3fc1f70 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:13:10.409830 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d99162dc-75f1-434f-8e7c-ae35f3fc1f70 service cinder] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:13:10.419497 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 930/1858] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jul 13 22:13:10 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 22:13:10.439930 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd62b3d-81a2-4a88-af47-79c313301d94 None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:13:10.491776 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:13:10.510955 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dcd62b3d-81a2-4a88-af47-79c313301d94 None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:13:10.521486 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6545fbdb-4e91-4d7a-96b2-07700aa09104 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:13:10.522425 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6545fbdb-4e91-4d7a-96b2-07700aa09104 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:13:10.523138 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6545fbdb-4e91-4d7a-96b2-07700aa09104 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:13:10.523138 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6545fbdb-4e91-4d7a-96b2-07700aa09104 admin admin] PATH_INFO: `/v3/users/c051bca299ec44e38c68e87674a7009d` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:13:10.524452 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6545fbdb-4e91-4d7a-96b2-07700aa09104 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c051bca299ec44e38c68e87674a7009d)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:13:10.526408 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6545fbdb-4e91-4d7a-96b2-07700aa09104 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:13:10.606397 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-6545fbdb-4e91-4d7a-96b2-07700aa09104 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c051bca299ec44e38c68e87674a7009d'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:13:10.616586 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-6545fbdb-4e91-4d7a-96b2-07700aa09104 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c051bca299ec44e38c68e87674a7009d'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:13:10.644517 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-6545fbdb-4e91-4d7a-96b2-07700aa09104 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c051bca299ec44e38c68e87674a7009d'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:13:10.645872 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-6545fbdb-4e91-4d7a-96b2-07700aa09104 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c051bca299ec44e38c68e87674a7009d'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:13:10.646710 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-6545fbdb-4e91-4d7a-96b2-07700aa09104 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c051bca299ec44e38c68e87674a7009d'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:13:10.662264 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 929/1859] 10.222.0.52 () {58 vars in 1223 bytes} [Wed Jul 13 22:13:10 2022] DELETE /identity/v3/users/c051bca299ec44e38c68e87674a7009d => generated 0 bytes in 223 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 22:13:10.668314 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4db865-eb0b-44e5-8cf9-31906b013448 None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:13:10.668759 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4db865-eb0b-44e5-8cf9-31906b013448 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:13:10.669207 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9a4db865-eb0b-44e5-8cf9-31906b013448 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:13:10.704761 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-9a4db865-eb0b-44e5-8cf9-31906b013448 None None] MFA Rules not processed for user `cc99c93da5074cf5a48bc2aa9a92c5e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 22:13:10.707028 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-9a4db865-eb0b-44e5-8cf9-31906b013448 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:13:10.742721 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:13:10.772969 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 931/1860] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jul 13 22:13:10 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 22:13:10.782944 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6545fbdb-4e91-4d7a-96b2-07700aa09104 admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:13:10.784444 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-6545fbdb-4e91-4d7a-96b2-07700aa09104 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:13:10.845053 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6545fbdb-4e91-4d7a-96b2-07700aa09104 admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:13:10.848804 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6c07c09c-b750-4803-916b-5b55734d08de service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '847ec9a1bd4d4dec804318a5c845895a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417e3e2f1333481795a089bef2e24bea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:13:10.849466 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c07c09c-b750-4803-916b-5b55734d08de service neutron] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:13:10.849767 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c07c09c-b750-4803-916b-5b55734d08de service neutron] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:13:10.850044 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6c07c09c-b750-4803-916b-5b55734d08de service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:13:10.860201 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c07c09c-b750-4803-916b-5b55734d08de service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:13:10.861672 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6c07c09c-b750-4803-916b-5b55734d08de service neutron] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:13:10.872014 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 930/1861] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jul 13 22:13:10 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 22:13:11.237741 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4db865-eb0b-44e5-8cf9-31906b013448 None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:13:11.239518 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-9a4db865-eb0b-44e5-8cf9-31906b013448 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:13:11.264357 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9a4db865-eb0b-44e5-8cf9-31906b013448 None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:13:11.270105 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e2f02b3-7a57-4c38-8277-9769ca866046 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:13:11.271067 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2f02b3-7a57-4c38-8277-9769ca866046 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:13:11.271414 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2f02b3-7a57-4c38-8277-9769ca866046 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:13:11.271658 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e2f02b3-7a57-4c38-8277-9769ca866046 admin admin] PATH_INFO: `/v3/projects/20056966506b46709e31c2f102451d1d` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:13:11.272944 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2f02b3-7a57-4c38-8277-9769ca866046 admin admin] RBAC: Authorizing `identity:delete_project(project_id=20056966506b46709e31c2f102451d1d)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:13:11.274921 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e2f02b3-7a57-4c38-8277-9769ca866046 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:13:11.326943 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-5e2f02b3-7a57-4c38-8277-9769ca866046 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '20056966506b46709e31c2f102451d1d', 'request_id': 'req-5e2f02b3-7a57-4c38-8277-9769ca866046'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:13:11.333946 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-5e2f02b3-7a57-4c38-8277-9769ca866046 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '20056966506b46709e31c2f102451d1d', 'request_id': 'req-5e2f02b3-7a57-4c38-8277-9769ca866046'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:13:11.340138 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-5e2f02b3-7a57-4c38-8277-9769ca866046 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '20056966506b46709e31c2f102451d1d', 'request_id': 'req-5e2f02b3-7a57-4c38-8277-9769ca866046'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:13:11.354082 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-5e2f02b3-7a57-4c38-8277-9769ca866046 admin admin] The token cache is being invalidate because project 20056966506b46709e31c2f102451d1d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80381) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 22:13:11.354365 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-5e2f02b3-7a57-4c38-8277-9769ca866046 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:13:11.355064 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-5e2f02b3-7a57-4c38-8277-9769ca866046 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:13:11.356690 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 932/1862] 10.222.0.52 () {58 vars in 1232 bytes} [Wed Jul 13 22:13:11 2022] DELETE /identity/v3/projects/20056966506b46709e31c2f102451d1d => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 22:13:11.361620 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd27d1c4-12f3-4ed3-a9b6-92e9dde86034 None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:13:11.362126 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd27d1c4-12f3-4ed3-a9b6-92e9dde86034 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:13:11.362202 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd27d1c4-12f3-4ed3-a9b6-92e9dde86034 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:13:11.382675 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-fd27d1c4-12f3-4ed3-a9b6-92e9dde86034 None None] MFA Rules not processed for user `cc99c93da5074cf5a48bc2aa9a92c5e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 22:13:11.383867 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-fd27d1c4-12f3-4ed3-a9b6-92e9dde86034 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:13:11.448365 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 931/1863] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jul 13 22:13:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 22:13:11.456353 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffd6071-b25a-4f47-a9eb-10bab5216751 None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:13:11.456862 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffd6071-b25a-4f47-a9eb-10bab5216751 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:13:11.456965 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ffd6071-b25a-4f47-a9eb-10bab5216751 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:13:11.475816 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-5ffd6071-b25a-4f47-a9eb-10bab5216751 None None] MFA Rules not processed for user `cc99c93da5074cf5a48bc2aa9a92c5e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 22:13:11.477412 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-5ffd6071-b25a-4f47-a9eb-10bab5216751 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:13:11.485336 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 933/1864] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jul 13 22:13:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 13 22:13:11.489678 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd27d1c4-12f3-4ed3-a9b6-92e9dde86034 None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:13:11.499774 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd27d1c4-12f3-4ed3-a9b6-92e9dde86034 None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:13:11.504808 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52edc954-7a83-4825-8025-24e084ca5eab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:13:11.505476 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52edc954-7a83-4825-8025-24e084ca5eab admin admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:13:11.505754 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52edc954-7a83-4825-8025-24e084ca5eab admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:13:11.506027 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52edc954-7a83-4825-8025-24e084ca5eab admin admin] PATH_INFO: `/v3/domains` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:13:11.506815 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52edc954-7a83-4825-8025-24e084ca5eab admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 22:13:11.507169 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52edc954-7a83-4825-8025-24e084ca5eab admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:13:11.508810 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52edc954-7a83-4825-8025-24e084ca5eab admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:13:11.511328 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:13:11.523701 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 932/1865] 10.222.0.52 () {58 vars in 1152 bytes} [Wed Jul 13 22:13:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 22:13:11.527923 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ffd6071-b25a-4f47-a9eb-10bab5216751 None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:13:11.537488 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ffd6071-b25a-4f47-a9eb-10bab5216751 None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:13:11.542568 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd551abe-10b1-4a89-8ddf-e6d4b4c2ea84 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:13:11.543448 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd551abe-10b1-4a89-8ddf-e6d4b4c2ea84 admin admin] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:13:11.543867 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd551abe-10b1-4a89-8ddf-e6d4b4c2ea84 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:13:11.544344 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd551abe-10b1-4a89-8ddf-e6d4b4c2ea84 admin admin] PATH_INFO: `/v3/projects` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:13:11.545201 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd551abe-10b1-4a89-8ddf-e6d4b4c2ea84 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:13:11.547540 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd551abe-10b1-4a89-8ddf-e6d4b4c2ea84 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:13:11.581529 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 934/1866] 10.222.0.52 () {60 vars in 1152 bytes} [Wed Jul 13 22:13:11 2022] POST /identity/v3/projects => generated 361 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 13 22:13:11.582831 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52edc954-7a83-4825-8025-24e084ca5eab admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:13:11.597827 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52edc954-7a83-4825-8025-24e084ca5eab admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:13:11.670073 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0d34ea-1f13-44d0-9a85-5e0e47b371da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:13:11.670828 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0d34ea-1f13-44d0-9a85-5e0e47b371da admin admin] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:13:11.671107 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0d34ea-1f13-44d0-9a85-5e0e47b371da admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:13:11.671873 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef0d34ea-1f13-44d0-9a85-5e0e47b371da admin admin] PATH_INFO: `/v3/users` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:13:11.672085 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef0d34ea-1f13-44d0-9a85-5e0e47b371da admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:13:11.674037 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef0d34ea-1f13-44d0-9a85-5e0e47b371da admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:13:11.700660 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 933/1867] 10.222.0.52 () {60 vars in 1143 bytes} [Wed Jul 13 22:13:11 2022] POST /identity/v3/users => generated 281 bytes in 116 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 22:13:11.702254 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd551abe-10b1-4a89-8ddf-e6d4b4c2ea84 admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:13:11.723467 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd551abe-10b1-4a89-8ddf-e6d4b4c2ea84 admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:13:11.729863 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5cf0e4-d920-4dc1-a56f-a2a28dbe7d20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:13:11.731605 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5cf0e4-d920-4dc1-a56f-a2a28dbe7d20 admin admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:13:11.732089 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5cf0e4-d920-4dc1-a56f-a2a28dbe7d20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:13:11.732557 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba5cf0e4-d920-4dc1-a56f-a2a28dbe7d20 admin admin] PATH_INFO: `/v3/roles` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:13:11.733744 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5cf0e4-d920-4dc1-a56f-a2a28dbe7d20 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:13:11.736108 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba5cf0e4-d920-4dc1-a56f-a2a28dbe7d20 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:13:11.770622 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 935/1868] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jul 13 22:13:11 2022] GET /identity/v3/roles => generated 1420 bytes in 67 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 22:13:11.772784 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0d34ea-1f13-44d0-9a85-5e0e47b371da admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:13:11.785452 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef0d34ea-1f13-44d0-9a85-5e0e47b371da admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:13:11.790664 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1dbae40c-07ef-4782-aed5-0ec13c3a58dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:13:11.791413 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbae40c-07ef-4782-aed5-0ec13c3a58dc admin admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:13:11.791697 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbae40c-07ef-4782-aed5-0ec13c3a58dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:13:11.791977 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1dbae40c-07ef-4782-aed5-0ec13c3a58dc admin admin] PATH_INFO: `/v3/roles` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:13:11.792736 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dbae40c-07ef-4782-aed5-0ec13c3a58dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:13:11.794432 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1dbae40c-07ef-4782-aed5-0ec13c3a58dc admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:13:11.816822 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 934/1869] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jul 13 22:13:11 2022] GET /identity/v3/roles => generated 1420 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 22:13:11.821163 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5cf0e4-d920-4dc1-a56f-a2a28dbe7d20 admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:13:11.833803 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba5cf0e4-d920-4dc1-a56f-a2a28dbe7d20 admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:13:11.839385 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b040c87d-35df-4c13-a05c-2969a76b7030 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:13:11.840290 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b040c87d-35df-4c13-a05c-2969a76b7030 admin admin] REQUEST_METHOD: `PUT` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:13:11.840574 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b040c87d-35df-4c13-a05c-2969a76b7030 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:13:11.841065 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b040c87d-35df-4c13-a05c-2969a76b7030 admin admin] PATH_INFO: `/v3/projects/fdc879746ebf45118639db81ec8f6c48/users/305db29aa9b44a23b4c148cff3431e7a/roles/2a5e86976394498abb6310697cdb3f8b` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:13:11.857318 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b040c87d-35df-4c13-a05c-2969a76b7030 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fdc879746ebf45118639db81ec8f6c48, user_id=305db29aa9b44a23b4c148cff3431e7a, role_id=2a5e86976394498abb6310697cdb3f8b)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:13:11.860033 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b040c87d-35df-4c13-a05c-2969a76b7030 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:13:11.885430 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 936/1870] 10.222.0.52 () {60 vars in 1482 bytes} [Wed Jul 13 22:13:11 2022] PUT /identity/v3/projects/fdc879746ebf45118639db81ec8f6c48/users/305db29aa9b44a23b4c148cff3431e7a/roles/2a5e86976394498abb6310697cdb3f8b => generated 0 bytes in 64 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 22:13:11.896365 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c450d075-4958-46de-b47f-7f0110668f3a None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:13:11.896720 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c450d075-4958-46de-b47f-7f0110668f3a None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:13:11.896954 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c450d075-4958-46de-b47f-7f0110668f3a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:13:11.931586 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-c450d075-4958-46de-b47f-7f0110668f3a None None] MFA Rules not processed for user `305db29aa9b44a23b4c148cff3431e7a`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 22:13:11.933052 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-c450d075-4958-46de-b47f-7f0110668f3a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:13:12.013301 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 935/1871] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jul 13 22:13:11 2022] POST /identity/v3/auth/tokens => generated 3228 bytes in 118 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 13 22:13:12.047403 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b040c87d-35df-4c13-a05c-2969a76b7030 admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:13:12.049245 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-b040c87d-35df-4c13-a05c-2969a76b7030 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:13:12.108978 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b040c87d-35df-4c13-a05c-2969a76b7030 admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:13:12.112781 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2614ab47-abea-427f-8e2d-1dc1aebd50e1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc77e1ea1ada4700b809beb1c980c5e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417e3e2f1333481795a089bef2e24bea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:13:12.113440 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2614ab47-abea-427f-8e2d-1dc1aebd50e1 service cinder] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:13:12.113722 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2614ab47-abea-427f-8e2d-1dc1aebd50e1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:13:12.114001 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2614ab47-abea-427f-8e2d-1dc1aebd50e1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:13:12.122878 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2614ab47-abea-427f-8e2d-1dc1aebd50e1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:13:12.124362 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2614ab47-abea-427f-8e2d-1dc1aebd50e1 service cinder] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:13:12.133513 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 937/1872] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jul 13 22:13:12 2022] GET /identity/v3/auth/tokens => generated 3228 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 22:13:12.147929 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c450d075-4958-46de-b47f-7f0110668f3a None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:13:12.214144 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c450d075-4958-46de-b47f-7f0110668f3a None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:13:12.219475 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025f39d3-8a0f-450b-b075-32659a154d0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:13:12.220366 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025f39d3-8a0f-450b-b075-32659a154d0d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:13:12.220638 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025f39d3-8a0f-450b-b075-32659a154d0d admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:13:12.220909 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-025f39d3-8a0f-450b-b075-32659a154d0d admin admin] PATH_INFO: `/v3/users/305db29aa9b44a23b4c148cff3431e7a` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:13:12.222120 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025f39d3-8a0f-450b-b075-32659a154d0d admin admin] RBAC: Authorizing `identity:delete_user(user_id=305db29aa9b44a23b4c148cff3431e7a)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:13:12.224079 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-025f39d3-8a0f-450b-b075-32659a154d0d admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:13:12.229530 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:13:12.289947 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-025f39d3-8a0f-450b-b075-32659a154d0d admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '305db29aa9b44a23b4c148cff3431e7a'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:13:12.300212 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-025f39d3-8a0f-450b-b075-32659a154d0d admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '305db29aa9b44a23b4c148cff3431e7a'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:13:12.313475 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-025f39d3-8a0f-450b-b075-32659a154d0d admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '305db29aa9b44a23b4c148cff3431e7a'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:13:12.314283 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-025f39d3-8a0f-450b-b075-32659a154d0d admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '305db29aa9b44a23b4c148cff3431e7a'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:13:12.315065 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-025f39d3-8a0f-450b-b075-32659a154d0d admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '305db29aa9b44a23b4c148cff3431e7a'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:13:12.328046 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 936/1873] 10.222.0.52 () {58 vars in 1223 bytes} [Wed Jul 13 22:13:12 2022] DELETE /identity/v3/users/305db29aa9b44a23b4c148cff3431e7a => generated 0 bytes in 181 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 22:13:12.332636 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad94d223-5cb4-4876-b7f9-911ba02bebb7 None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:13:12.332998 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad94d223-5cb4-4876-b7f9-911ba02bebb7 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:13:12.333343 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad94d223-5cb4-4876-b7f9-911ba02bebb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:13:12.339426 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:13:12.352817 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-ad94d223-5cb4-4876-b7f9-911ba02bebb7 None None] MFA Rules not processed for user `cc99c93da5074cf5a48bc2aa9a92c5e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 22:13:12.354512 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-ad94d223-5cb4-4876-b7f9-911ba02bebb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:13:12.415586 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 938/1874] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jul 13 22:13:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 13 22:13:12.425947 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025f39d3-8a0f-450b-b075-32659a154d0d admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:13:12.427422 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-025f39d3-8a0f-450b-b075-32659a154d0d admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:13:12.487809 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-025f39d3-8a0f-450b-b075-32659a154d0d admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:13:12.491629 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e94be005-7bc3-40c8-8eff-5dd4acbc484b service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '847ec9a1bd4d4dec804318a5c845895a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417e3e2f1333481795a089bef2e24bea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:13:12.492684 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94be005-7bc3-40c8-8eff-5dd4acbc484b service neutron] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:13:12.492783 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94be005-7bc3-40c8-8eff-5dd4acbc484b service neutron] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:13:12.492988 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e94be005-7bc3-40c8-8eff-5dd4acbc484b service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:13:12.504688 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94be005-7bc3-40c8-8eff-5dd4acbc484b service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:13:12.507073 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e94be005-7bc3-40c8-8eff-5dd4acbc484b service neutron] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:13:12.525279 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 937/1875] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jul 13 22:13:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 22:13:12.839011 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad94d223-5cb4-4876-b7f9-911ba02bebb7 None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:13:12.845896 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-ad94d223-5cb4-4876-b7f9-911ba02bebb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:13:12.868872 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad94d223-5cb4-4876-b7f9-911ba02bebb7 None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:13:12.885796 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd7bdc79-cdaa-48b5-b4ea-3e2dfcf2a6b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:13:12.886814 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7bdc79-cdaa-48b5-b4ea-3e2dfcf2a6b7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:13:12.887091 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7bdc79-cdaa-48b5-b4ea-3e2dfcf2a6b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:13:12.887462 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd7bdc79-cdaa-48b5-b4ea-3e2dfcf2a6b7 admin admin] PATH_INFO: `/v3/projects/fdc879746ebf45118639db81ec8f6c48` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:13:12.891274 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7bdc79-cdaa-48b5-b4ea-3e2dfcf2a6b7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fdc879746ebf45118639db81ec8f6c48)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:13:12.893328 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd7bdc79-cdaa-48b5-b4ea-3e2dfcf2a6b7 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:13:12.975855 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-cd7bdc79-cdaa-48b5-b4ea-3e2dfcf2a6b7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fdc879746ebf45118639db81ec8f6c48', 'request_id': 'req-cd7bdc79-cdaa-48b5-b4ea-3e2dfcf2a6b7'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:13:12.983563 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-cd7bdc79-cdaa-48b5-b4ea-3e2dfcf2a6b7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fdc879746ebf45118639db81ec8f6c48', 'request_id': 'req-cd7bdc79-cdaa-48b5-b4ea-3e2dfcf2a6b7'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:13:12.992529 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-cd7bdc79-cdaa-48b5-b4ea-3e2dfcf2a6b7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fdc879746ebf45118639db81ec8f6c48', 'request_id': 'req-cd7bdc79-cdaa-48b5-b4ea-3e2dfcf2a6b7'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:13:13.014530 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-cd7bdc79-cdaa-48b5-b4ea-3e2dfcf2a6b7 admin admin] The token cache is being invalidate because project fdc879746ebf45118639db81ec8f6c48 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80381) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 22:13:13.015081 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-cd7bdc79-cdaa-48b5-b4ea-3e2dfcf2a6b7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:13:13.015910 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-cd7bdc79-cdaa-48b5-b4ea-3e2dfcf2a6b7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:13:13.017617 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 939/1876] 10.222.0.52 () {58 vars in 1232 bytes} [Wed Jul 13 22:13:12 2022] DELETE /identity/v3/projects/fdc879746ebf45118639db81ec8f6c48 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 22:13:13.023136 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed1edbf-839c-43fe-8873-35a7b0849444 None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:13:13.023580 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed1edbf-839c-43fe-8873-35a7b0849444 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:13:13.023949 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3ed1edbf-839c-43fe-8873-35a7b0849444 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:13:13.047063 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-3ed1edbf-839c-43fe-8873-35a7b0849444 None None] MFA Rules not processed for user `cc99c93da5074cf5a48bc2aa9a92c5e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 22:13:13.048298 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-3ed1edbf-839c-43fe-8873-35a7b0849444 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:13:13.113696 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 938/1877] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jul 13 22:13:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 13 22:13:13.126555 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54c67d3-81c6-4960-997a-201ba8c5cf7d None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:13:13.126850 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54c67d3-81c6-4960-997a-201ba8c5cf7d None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:13:13.127129 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d54c67d3-81c6-4960-997a-201ba8c5cf7d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:13:13.146024 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-d54c67d3-81c6-4960-997a-201ba8c5cf7d None None] MFA Rules not processed for user `cc99c93da5074cf5a48bc2aa9a92c5e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 22:13:13.147263 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-d54c67d3-81c6-4960-997a-201ba8c5cf7d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:13:13.155505 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 940/1878] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jul 13 22:13:13 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 22:13:13.159977 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed1edbf-839c-43fe-8873-35a7b0849444 None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:13:13.172192 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3ed1edbf-839c-43fe-8873-35a7b0849444 None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:13:13.177322 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e7382f-3c1a-4499-afdb-4ea258e5a6a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:13:13.178007 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e7382f-3c1a-4499-afdb-4ea258e5a6a4 admin admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:13:13.178291 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e7382f-3c1a-4499-afdb-4ea258e5a6a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:13:13.178590 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1e7382f-3c1a-4499-afdb-4ea258e5a6a4 admin admin] PATH_INFO: `/v3/domains` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:13:13.179489 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e7382f-3c1a-4499-afdb-4ea258e5a6a4 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 22:13:13.179798 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e7382f-3c1a-4499-afdb-4ea258e5a6a4 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:13:13.181416 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1e7382f-3c1a-4499-afdb-4ea258e5a6a4 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:13:13.196472 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 939/1879] 10.222.0.52 () {58 vars in 1152 bytes} [Wed Jul 13 22:13:13 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 22:13:13.201300 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54c67d3-81c6-4960-997a-201ba8c5cf7d None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:13:13.212518 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d54c67d3-81c6-4960-997a-201ba8c5cf7d None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:13:13.217435 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0574b89-440f-4f00-98ef-2a799a6f3a75 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:13:13.218111 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0574b89-440f-4f00-98ef-2a799a6f3a75 admin admin] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:13:13.218387 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0574b89-440f-4f00-98ef-2a799a6f3a75 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:13:13.218691 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0574b89-440f-4f00-98ef-2a799a6f3a75 admin admin] PATH_INFO: `/v3/projects` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:13:13.219393 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0574b89-440f-4f00-98ef-2a799a6f3a75 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:13:13.221328 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0574b89-440f-4f00-98ef-2a799a6f3a75 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:13:13.265540 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 941/1880] 10.222.0.52 () {60 vars in 1152 bytes} [Wed Jul 13 22:13:13 2022] POST /identity/v3/projects => generated 385 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 22:13:13.266275 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e7382f-3c1a-4499-afdb-4ea258e5a6a4 admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:13:13.283972 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1e7382f-3c1a-4499-afdb-4ea258e5a6a4 admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:13:13.287761 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:13:13.348860 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67bc9fdd-d78c-4c79-aec7-8388bd200850 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:13:13.349586 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bc9fdd-d78c-4c79-aec7-8388bd200850 admin admin] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:13:13.349904 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bc9fdd-d78c-4c79-aec7-8388bd200850 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:13:13.350181 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67bc9fdd-d78c-4c79-aec7-8388bd200850 admin admin] PATH_INFO: `/v3/users` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:13:13.351090 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67bc9fdd-d78c-4c79-aec7-8388bd200850 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:13:13.353455 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67bc9fdd-d78c-4c79-aec7-8388bd200850 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:13:13.383089 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 940/1881] 10.222.0.52 () {60 vars in 1143 bytes} [Wed Jul 13 22:13:13 2022] POST /identity/v3/users => generated 293 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 13 22:13:13.387417 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0574b89-440f-4f00-98ef-2a799a6f3a75 admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:13:13.398309 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0574b89-440f-4f00-98ef-2a799a6f3a75 admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:13:13.402650 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe2ac32-fe56-49cc-b51d-59510ad7dae1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:13:13.403293 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe2ac32-fe56-49cc-b51d-59510ad7dae1 admin admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:13:13.403576 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe2ac32-fe56-49cc-b51d-59510ad7dae1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:13:13.403849 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbe2ac32-fe56-49cc-b51d-59510ad7dae1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:13:13.404581 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe2ac32-fe56-49cc-b51d-59510ad7dae1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:13:13.406208 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbe2ac32-fe56-49cc-b51d-59510ad7dae1 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:13:13.417840 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 942/1882] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jul 13 22:13:13 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 13 22:13:13.421607 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67bc9fdd-d78c-4c79-aec7-8388bd200850 admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:13:13.431933 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67bc9fdd-d78c-4c79-aec7-8388bd200850 admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:13:13.436729 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4d86168-7949-4472-a401-91c44c1aca52 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:13:13.437405 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d86168-7949-4472-a401-91c44c1aca52 admin admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:13:13.437682 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d86168-7949-4472-a401-91c44c1aca52 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:13:13.437952 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4d86168-7949-4472-a401-91c44c1aca52 admin admin] PATH_INFO: `/v3/roles` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:13:13.438714 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d86168-7949-4472-a401-91c44c1aca52 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:13:13.440371 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4d86168-7949-4472-a401-91c44c1aca52 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:13:13.450965 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 941/1883] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jul 13 22:13:13 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 22:13:13.455132 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe2ac32-fe56-49cc-b51d-59510ad7dae1 admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:13:13.465680 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbe2ac32-fe56-49cc-b51d-59510ad7dae1 admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:13:13.470351 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe672175-c54b-4ff5-a509-853b103cf3b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:13:13.471356 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe672175-c54b-4ff5-a509-853b103cf3b3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:13:13.471694 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe672175-c54b-4ff5-a509-853b103cf3b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:13:13.472005 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe672175-c54b-4ff5-a509-853b103cf3b3 admin admin] PATH_INFO: `/v3/projects/ed77704452c046358a5b729098a3e9cc/users/5090733633d04eafaa0349f9f04d6781/roles/2a5e86976394498abb6310697cdb3f8b` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:13:13.488756 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe672175-c54b-4ff5-a509-853b103cf3b3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ed77704452c046358a5b729098a3e9cc, user_id=5090733633d04eafaa0349f9f04d6781, role_id=2a5e86976394498abb6310697cdb3f8b)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:13:13.491980 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe672175-c54b-4ff5-a509-853b103cf3b3 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:13:13.506937 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 943/1884] 10.222.0.52 () {60 vars in 1482 bytes} [Wed Jul 13 22:13:13 2022] PUT /identity/v3/projects/ed77704452c046358a5b729098a3e9cc/users/5090733633d04eafaa0349f9f04d6781/roles/2a5e86976394498abb6310697cdb3f8b => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 13 22:13:13.517737 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34664d0c-135a-4889-9e03-06330ce8b165 None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:13:13.518020 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34664d0c-135a-4889-9e03-06330ce8b165 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:13:13.518377 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34664d0c-135a-4889-9e03-06330ce8b165 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:13:13.551682 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-34664d0c-135a-4889-9e03-06330ce8b165 None None] MFA Rules not processed for user `5090733633d04eafaa0349f9f04d6781`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 22:13:13.553162 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-34664d0c-135a-4889-9e03-06330ce8b165 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:13:13.623484 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 942/1885] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jul 13 22:13:13 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 22:13:13.628205 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe672175-c54b-4ff5-a509-853b103cf3b3 admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:13:13.653936 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:13:13.695024 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe672175-c54b-4ff5-a509-853b103cf3b3 admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:13:13.700466 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a702c858-29d0-4ea5-8e5c-870daf5a3e37 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:13:13.701180 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a702c858-29d0-4ea5-8e5c-870daf5a3e37 admin admin] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:13:13.701458 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a702c858-29d0-4ea5-8e5c-870daf5a3e37 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:13:13.701729 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a702c858-29d0-4ea5-8e5c-870daf5a3e37 admin admin] PATH_INFO: `/v3/projects` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:13:13.702440 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a702c858-29d0-4ea5-8e5c-870daf5a3e37 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:13:13.704588 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a702c858-29d0-4ea5-8e5c-870daf5a3e37 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:13:13.733266 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 944/1886] 10.222.0.52 () {60 vars in 1152 bytes} [Wed Jul 13 22:13:13 2022] POST /identity/v3/projects => generated 383 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 22:13:13.737698 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34664d0c-135a-4889-9e03-06330ce8b165 None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:13:13.749499 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34664d0c-135a-4889-9e03-06330ce8b165 None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:13:13.814066 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a0dc93-5381-412b-ad76-288b479e4c60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:13:13.814847 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a0dc93-5381-412b-ad76-288b479e4c60 admin admin] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:13:13.815131 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a0dc93-5381-412b-ad76-288b479e4c60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:13:13.815410 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96a0dc93-5381-412b-ad76-288b479e4c60 admin admin] PATH_INFO: `/v3/users` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:13:13.816177 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a0dc93-5381-412b-ad76-288b479e4c60 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:13:13.818126 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96a0dc93-5381-412b-ad76-288b479e4c60 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:13:13.840771 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 943/1887] 10.222.0.52 () {60 vars in 1143 bytes} [Wed Jul 13 22:13:13 2022] POST /identity/v3/users => generated 292 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 22:13:13.844968 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a702c858-29d0-4ea5-8e5c-870daf5a3e37 admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:13:13.857496 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a702c858-29d0-4ea5-8e5c-870daf5a3e37 admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:13:13.862462 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59a8137-bf35-4a82-b24c-c94bca7a984b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:13:13.863217 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59a8137-bf35-4a82-b24c-c94bca7a984b admin admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:13:13.863520 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59a8137-bf35-4a82-b24c-c94bca7a984b admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:13:13.863776 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a59a8137-bf35-4a82-b24c-c94bca7a984b admin admin] PATH_INFO: `/v3/roles` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:13:13.864528 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59a8137-bf35-4a82-b24c-c94bca7a984b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:13:13.866158 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a59a8137-bf35-4a82-b24c-c94bca7a984b admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:13:13.877663 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 945/1888] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jul 13 22:13:13 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 22:13:13.881743 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a0dc93-5381-412b-ad76-288b479e4c60 admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:13:13.892551 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96a0dc93-5381-412b-ad76-288b479e4c60 admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:13:13.897334 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-38fac04a-7f84-4491-9357-9e8108712e6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:13:13.898305 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fac04a-7f84-4491-9357-9e8108712e6d admin admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:13:13.898466 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fac04a-7f84-4491-9357-9e8108712e6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:13:13.898817 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-38fac04a-7f84-4491-9357-9e8108712e6d admin admin] PATH_INFO: `/v3/roles` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:13:13.899588 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38fac04a-7f84-4491-9357-9e8108712e6d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:13:13.901445 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-38fac04a-7f84-4491-9357-9e8108712e6d admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:13:13.907708 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:13:13.913807 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 944/1889] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jul 13 22:13:13 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 22:13:13.918918 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59a8137-bf35-4a82-b24c-c94bca7a984b admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:13:13.929943 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a59a8137-bf35-4a82-b24c-c94bca7a984b admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:13:13.935427 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da55c61-f143-482c-8d56-f49ca89712f9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:13:13.937246 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da55c61-f143-482c-8d56-f49ca89712f9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:13:13.937735 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da55c61-f143-482c-8d56-f49ca89712f9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:13:13.938072 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7da55c61-f143-482c-8d56-f49ca89712f9 admin admin] PATH_INFO: `/v3/projects/36a974d17ddb4459b1b48d8b2536d5f5/users/fa1dc9833458430f8b7d8c4a77fd088c/roles/2a5e86976394498abb6310697cdb3f8b` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:13:13.955909 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da55c61-f143-482c-8d56-f49ca89712f9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=36a974d17ddb4459b1b48d8b2536d5f5, user_id=fa1dc9833458430f8b7d8c4a77fd088c, role_id=2a5e86976394498abb6310697cdb3f8b)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:13:13.958670 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7da55c61-f143-482c-8d56-f49ca89712f9 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:13:13.973965 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 946/1890] 10.222.0.52 () {60 vars in 1482 bytes} [Wed Jul 13 22:13:13 2022] PUT /identity/v3/projects/36a974d17ddb4459b1b48d8b2536d5f5/users/fa1dc9833458430f8b7d8c4a77fd088c/roles/2a5e86976394498abb6310697cdb3f8b => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 22:13:13.985993 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5acc23-b9ac-4367-8638-c49366e35243 None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:13:13.986375 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5acc23-b9ac-4367-8638-c49366e35243 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:13:13.986744 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f5acc23-b9ac-4367-8638-c49366e35243 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:13:14.014889 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-2f5acc23-b9ac-4367-8638-c49366e35243 None None] MFA Rules not processed for user `fa1dc9833458430f8b7d8c4a77fd088c`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 22:13:14.016110 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-2f5acc23-b9ac-4367-8638-c49366e35243 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:13:14.087542 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 945/1891] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jul 13 22:13:13 2022] POST /identity/v3/auth/tokens => generated 3250 bytes in 103 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 22:13:14.092247 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da55c61-f143-482c-8d56-f49ca89712f9 admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:13:14.178277 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7da55c61-f143-482c-8d56-f49ca89712f9 admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:13:14.184419 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274eedba-94e1-4a7d-87d9-a5ad0af70632 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:13:14.185739 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274eedba-94e1-4a7d-87d9-a5ad0af70632 admin admin] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:13:14.186071 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274eedba-94e1-4a7d-87d9-a5ad0af70632 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:13:14.186723 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-274eedba-94e1-4a7d-87d9-a5ad0af70632 admin admin] PATH_INFO: `/v3/projects` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:13:14.187752 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274eedba-94e1-4a7d-87d9-a5ad0af70632 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:13:14.189948 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-274eedba-94e1-4a7d-87d9-a5ad0af70632 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:13:14.226538 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 947/1892] 10.222.0.52 () {60 vars in 1152 bytes} [Wed Jul 13 22:13:14 2022] POST /identity/v3/projects => generated 383 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 22:13:14.228818 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5acc23-b9ac-4367-8638-c49366e35243 None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:13:14.240459 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f5acc23-b9ac-4367-8638-c49366e35243 None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:13:14.312295 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c2ad76-d15f-43cc-8930-ca54b46319b7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:13:14.313315 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c2ad76-d15f-43cc-8930-ca54b46319b7 admin admin] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:13:14.313315 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c2ad76-d15f-43cc-8930-ca54b46319b7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:13:14.313497 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05c2ad76-d15f-43cc-8930-ca54b46319b7 admin admin] PATH_INFO: `/v3/users` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:13:14.314199 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c2ad76-d15f-43cc-8930-ca54b46319b7 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:13:14.316371 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05c2ad76-d15f-43cc-8930-ca54b46319b7 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:13:14.345061 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 946/1893] 10.222.0.52 () {60 vars in 1143 bytes} [Wed Jul 13 22:13:14 2022] POST /identity/v3/users => generated 298 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 22:13:14.346699 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274eedba-94e1-4a7d-87d9-a5ad0af70632 admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:13:14.361511 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-274eedba-94e1-4a7d-87d9-a5ad0af70632 admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:13:14.368735 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646fdc08-31d4-4245-98d2-8c76a99359bb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:13:14.370267 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646fdc08-31d4-4245-98d2-8c76a99359bb admin admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:13:14.370953 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646fdc08-31d4-4245-98d2-8c76a99359bb admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:13:14.371434 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-646fdc08-31d4-4245-98d2-8c76a99359bb admin admin] PATH_INFO: `/v3/roles` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:13:14.382210 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646fdc08-31d4-4245-98d2-8c76a99359bb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:13:14.387494 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-646fdc08-31d4-4245-98d2-8c76a99359bb admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:13:14.405291 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 948/1894] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jul 13 22:13:14 2022] GET /identity/v3/roles => generated 1420 bytes in 56 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 22:13:14.414759 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c2ad76-d15f-43cc-8930-ca54b46319b7 admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:13:14.427228 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05c2ad76-d15f-43cc-8930-ca54b46319b7 admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:13:14.435108 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be891482-c340-461b-adb0-91a0e191bd4e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:13:14.439026 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be891482-c340-461b-adb0-91a0e191bd4e admin admin] REQUEST_METHOD: `PUT` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:13:14.439486 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be891482-c340-461b-adb0-91a0e191bd4e admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:13:14.439907 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be891482-c340-461b-adb0-91a0e191bd4e admin admin] PATH_INFO: `/v3/projects/3e340dd42cf540d2ad001b379f71250a/users/c678fa74aac84152afbedfc4b8ba7de9/roles/44efba2579f342df8a0789c59d163ec3` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:13:14.460086 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be891482-c340-461b-adb0-91a0e191bd4e admin admin] RBAC: Authorizing `identity:create_grant(project_id=3e340dd42cf540d2ad001b379f71250a, user_id=c678fa74aac84152afbedfc4b8ba7de9, role_id=44efba2579f342df8a0789c59d163ec3)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:13:14.463012 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be891482-c340-461b-adb0-91a0e191bd4e admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:13:14.483617 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 947/1895] 10.222.0.52 () {60 vars in 1482 bytes} [Wed Jul 13 22:13:14 2022] PUT /identity/v3/projects/3e340dd42cf540d2ad001b379f71250a/users/c678fa74aac84152afbedfc4b8ba7de9/roles/44efba2579f342df8a0789c59d163ec3 => generated 0 bytes in 69 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 22:13:14.487444 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646fdc08-31d4-4245-98d2-8c76a99359bb admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:13:14.502852 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-646fdc08-31d4-4245-98d2-8c76a99359bb admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:13:14.552777 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:13:14.567738 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-823ec8fe-7192-4e1b-8ca0-c782b00714d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:13:14.568392 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823ec8fe-7192-4e1b-8ca0-c782b00714d2 admin admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:13:14.568678 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823ec8fe-7192-4e1b-8ca0-c782b00714d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:13:14.568954 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-823ec8fe-7192-4e1b-8ca0-c782b00714d2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:13:14.569726 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-823ec8fe-7192-4e1b-8ca0-c782b00714d2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:13:14.571373 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-823ec8fe-7192-4e1b-8ca0-c782b00714d2 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:13:14.582234 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 949/1896] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jul 13 22:13:14 2022] GET /identity/v3/roles => generated 1420 bytes in 95 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 22:13:14.586704 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be891482-c340-461b-adb0-91a0e191bd4e admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:13:14.597032 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be891482-c340-461b-adb0-91a0e191bd4e admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:13:14.601832 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe6a0f3-2907-4094-9e70-c238baa56ecf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:13:14.602762 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe6a0f3-2907-4094-9e70-c238baa56ecf admin admin] REQUEST_METHOD: `PUT` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:13:14.603038 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe6a0f3-2907-4094-9e70-c238baa56ecf admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:13:14.603308 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bbe6a0f3-2907-4094-9e70-c238baa56ecf admin admin] PATH_INFO: `/v3/projects/3e340dd42cf540d2ad001b379f71250a/users/c678fa74aac84152afbedfc4b8ba7de9/roles/44efba2579f342df8a0789c59d163ec3` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:13:14.605388 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe6a0f3-2907-4094-9e70-c238baa56ecf admin admin] RBAC: Authorizing `identity:create_grant(project_id=3e340dd42cf540d2ad001b379f71250a, user_id=c678fa74aac84152afbedfc4b8ba7de9, role_id=44efba2579f342df8a0789c59d163ec3)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:13:14.608036 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bbe6a0f3-2907-4094-9e70-c238baa56ecf admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:13:14.618739 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 948/1897] 10.222.0.52 () {60 vars in 1482 bytes} [Wed Jul 13 22:13:14 2022] PUT /identity/v3/projects/3e340dd42cf540d2ad001b379f71250a/users/c678fa74aac84152afbedfc4b8ba7de9/roles/44efba2579f342df8a0789c59d163ec3 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 22:13:14.628929 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e4ff29-c5b4-4063-8e0b-790cdd39fbf0 None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:13:14.629240 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e4ff29-c5b4-4063-8e0b-790cdd39fbf0 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:13:14.629528 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e4ff29-c5b4-4063-8e0b-790cdd39fbf0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:13:14.656305 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-a7e4ff29-c5b4-4063-8e0b-790cdd39fbf0 None None] MFA Rules not processed for user `c678fa74aac84152afbedfc4b8ba7de9`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 22:13:14.657516 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-a7e4ff29-c5b4-4063-8e0b-790cdd39fbf0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:13:14.732608 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 950/1898] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jul 13 22:13:14 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 22:13:14.744077 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe6a0f3-2907-4094-9e70-c238baa56ecf admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:13:14.746102 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-bbe6a0f3-2907-4094-9e70-c238baa56ecf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:13:14.795768 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:13:14.808349 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bbe6a0f3-2907-4094-9e70-c238baa56ecf admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:13:14.812430 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f7647f-8a18-4da4-8a5d-d405a5153645 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc77e1ea1ada4700b809beb1c980c5e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417e3e2f1333481795a089bef2e24bea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:13:14.813108 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f7647f-8a18-4da4-8a5d-d405a5153645 service cinder] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:13:14.813389 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f7647f-8a18-4da4-8a5d-d405a5153645 service cinder] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:13:14.813663 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f0f7647f-8a18-4da4-8a5d-d405a5153645 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:13:14.871761 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f7647f-8a18-4da4-8a5d-d405a5153645 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:13:14.873210 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f0f7647f-8a18-4da4-8a5d-d405a5153645 service cinder] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:13:14.882676 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 949/1899] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jul 13 22:13:14 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 139 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 22:13:17.461023 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e4ff29-c5b4-4063-8e0b-790cdd39fbf0 None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:13:17.462720 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-a7e4ff29-c5b4-4063-8e0b-790cdd39fbf0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:13:17.476867 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e4ff29-c5b4-4063-8e0b-790cdd39fbf0 None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:13:17.482937 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138dd4c0-0308-4ab8-8e58-4db2de394a1a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc77e1ea1ada4700b809beb1c980c5e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417e3e2f1333481795a089bef2e24bea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:13:17.483849 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138dd4c0-0308-4ab8-8e58-4db2de394a1a service cinder] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:13:17.484179 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138dd4c0-0308-4ab8-8e58-4db2de394a1a service cinder] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:13:17.484510 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138dd4c0-0308-4ab8-8e58-4db2de394a1a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:13:17.547218 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138dd4c0-0308-4ab8-8e58-4db2de394a1a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:13:17.548679 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138dd4c0-0308-4ab8-8e58-4db2de394a1a service cinder] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:13:17.558540 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 951/1900] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jul 13 22:13:17 2022] GET /identity/v3/auth/tokens => generated 3250 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 22:13:18.821067 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f7647f-8a18-4da4-8a5d-d405a5153645 service cinder] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:13:18.830596 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f0f7647f-8a18-4da4-8a5d-d405a5153645 service cinder] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:13:18.835338 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f723499b-c823-44d0-ac32-f916f2ef2806 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc77e1ea1ada4700b809beb1c980c5e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417e3e2f1333481795a089bef2e24bea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:13:18.835977 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f723499b-c823-44d0-ac32-f916f2ef2806 service cinder] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:13:18.836405 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f723499b-c823-44d0-ac32-f916f2ef2806 service cinder] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:13:18.836804 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f723499b-c823-44d0-ac32-f916f2ef2806 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:13:18.846125 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f723499b-c823-44d0-ac32-f916f2ef2806 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:13:18.847622 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f723499b-c823-44d0-ac32-f916f2ef2806 service cinder] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:13:18.857328 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 950/1901] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jul 13 22:13:18 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 22:13:25.972539 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138dd4c0-0308-4ab8-8e58-4db2de394a1a service cinder] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:13:26.039677 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138dd4c0-0308-4ab8-8e58-4db2de394a1a service cinder] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:13:26.045202 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8caf306-b555-4d22-bf58-8f155fd27d2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:13:26.046119 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8caf306-b555-4d22-bf58-8f155fd27d2f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:13:26.046398 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8caf306-b555-4d22-bf58-8f155fd27d2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:13:26.046704 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8caf306-b555-4d22-bf58-8f155fd27d2f admin admin] PATH_INFO: `/v3/users/5090733633d04eafaa0349f9f04d6781` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:13:26.048035 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8caf306-b555-4d22-bf58-8f155fd27d2f admin admin] RBAC: Authorizing `identity:delete_user(user_id=5090733633d04eafaa0349f9f04d6781)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:13:26.050123 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8caf306-b555-4d22-bf58-8f155fd27d2f admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:13:26.058150 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:13:26.117826 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-b8caf306-b555-4d22-bf58-8f155fd27d2f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5090733633d04eafaa0349f9f04d6781'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:13:26.127996 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-b8caf306-b555-4d22-bf58-8f155fd27d2f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5090733633d04eafaa0349f9f04d6781'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:13:26.143189 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-b8caf306-b555-4d22-bf58-8f155fd27d2f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5090733633d04eafaa0349f9f04d6781'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:13:26.144086 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-b8caf306-b555-4d22-bf58-8f155fd27d2f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5090733633d04eafaa0349f9f04d6781'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:13:26.144777 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-b8caf306-b555-4d22-bf58-8f155fd27d2f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5090733633d04eafaa0349f9f04d6781'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:13:26.162236 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 952/1902] 10.222.0.52 () {58 vars in 1223 bytes} [Wed Jul 13 22:13:25 2022] DELETE /identity/v3/users/5090733633d04eafaa0349f9f04d6781 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 22:13:26.167058 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9890d08-4b13-4795-ad09-d0e0aaf8c541 None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:13:26.167610 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9890d08-4b13-4795-ad09-d0e0aaf8c541 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:13:26.167702 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e9890d08-4b13-4795-ad09-d0e0aaf8c541 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:13:26.187095 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-e9890d08-4b13-4795-ad09-d0e0aaf8c541 None None] MFA Rules not processed for user `cc99c93da5074cf5a48bc2aa9a92c5e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 22:13:26.188420 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-e9890d08-4b13-4795-ad09-d0e0aaf8c541 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:13:26.266356 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 951/1903] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jul 13 22:13:26 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 13 22:13:26.276153 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8caf306-b555-4d22-bf58-8f155fd27d2f admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:13:26.277588 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-b8caf306-b555-4d22-bf58-8f155fd27d2f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:13:26.336851 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8caf306-b555-4d22-bf58-8f155fd27d2f admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:13:26.340495 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db36ac94-ea59-4960-ad1b-0269d60a9416 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '847ec9a1bd4d4dec804318a5c845895a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417e3e2f1333481795a089bef2e24bea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:13:26.341279 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db36ac94-ea59-4960-ad1b-0269d60a9416 service neutron] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:13:26.341565 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db36ac94-ea59-4960-ad1b-0269d60a9416 service neutron] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:13:26.341840 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db36ac94-ea59-4960-ad1b-0269d60a9416 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:13:26.351239 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db36ac94-ea59-4960-ad1b-0269d60a9416 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:13:26.352722 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db36ac94-ea59-4960-ad1b-0269d60a9416 service neutron] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:13:26.362752 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 953/1904] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jul 13 22:13:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 22:13:26.643093 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9890d08-4b13-4795-ad09-d0e0aaf8c541 None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:13:26.645093 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-e9890d08-4b13-4795-ad09-d0e0aaf8c541 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:13:26.660448 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e9890d08-4b13-4795-ad09-d0e0aaf8c541 None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:13:26.665456 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ab57e4-9bbe-4fa9-a885-ef0eaafd16e7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:13:26.666755 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ab57e4-9bbe-4fa9-a885-ef0eaafd16e7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:13:26.667026 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ab57e4-9bbe-4fa9-a885-ef0eaafd16e7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:13:26.667287 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60ab57e4-9bbe-4fa9-a885-ef0eaafd16e7 admin admin] PATH_INFO: `/v3/projects/ed77704452c046358a5b729098a3e9cc` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:13:26.668506 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ab57e4-9bbe-4fa9-a885-ef0eaafd16e7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ed77704452c046358a5b729098a3e9cc)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:13:26.670665 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60ab57e4-9bbe-4fa9-a885-ef0eaafd16e7 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:13:26.727624 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-60ab57e4-9bbe-4fa9-a885-ef0eaafd16e7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ed77704452c046358a5b729098a3e9cc', 'request_id': 'req-60ab57e4-9bbe-4fa9-a885-ef0eaafd16e7'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:13:26.734541 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-60ab57e4-9bbe-4fa9-a885-ef0eaafd16e7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ed77704452c046358a5b729098a3e9cc', 'request_id': 'req-60ab57e4-9bbe-4fa9-a885-ef0eaafd16e7'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:13:26.740724 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-60ab57e4-9bbe-4fa9-a885-ef0eaafd16e7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ed77704452c046358a5b729098a3e9cc', 'request_id': 'req-60ab57e4-9bbe-4fa9-a885-ef0eaafd16e7'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:13:26.761380 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-60ab57e4-9bbe-4fa9-a885-ef0eaafd16e7 admin admin] The token cache is being invalidate because project ed77704452c046358a5b729098a3e9cc was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80382) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 22:13:26.761675 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-60ab57e4-9bbe-4fa9-a885-ef0eaafd16e7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:13:26.762381 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-60ab57e4-9bbe-4fa9-a885-ef0eaafd16e7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:13:26.764178 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 952/1905] 10.222.0.52 () {58 vars in 1232 bytes} [Wed Jul 13 22:13:26 2022] DELETE /identity/v3/projects/ed77704452c046358a5b729098a3e9cc => generated 0 bytes in 122 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 22:13:26.768193 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db36ac94-ea59-4960-ad1b-0269d60a9416 service neutron] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:13:26.780756 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db36ac94-ea59-4960-ad1b-0269d60a9416 service neutron] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:13:26.838276 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd033ee-2431-4547-bfe2-915b86266b45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:13:26.839078 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd033ee-2431-4547-bfe2-915b86266b45 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:13:26.839359 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd033ee-2431-4547-bfe2-915b86266b45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:13:26.839628 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cd033ee-2431-4547-bfe2-915b86266b45 admin admin] PATH_INFO: `/v3/users/fa1dc9833458430f8b7d8c4a77fd088c` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:13:26.840808 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd033ee-2431-4547-bfe2-915b86266b45 admin admin] RBAC: Authorizing `identity:delete_user(user_id=fa1dc9833458430f8b7d8c4a77fd088c)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:13:26.843060 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9cd033ee-2431-4547-bfe2-915b86266b45 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:13:26.881544 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=84,keystone:DELETE=15,keystone:INSERT=1 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:13:26.897701 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-9cd033ee-2431-4547-bfe2-915b86266b45 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fa1dc9833458430f8b7d8c4a77fd088c'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:13:26.910302 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-9cd033ee-2431-4547-bfe2-915b86266b45 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fa1dc9833458430f8b7d8c4a77fd088c'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:13:26.924624 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-9cd033ee-2431-4547-bfe2-915b86266b45 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fa1dc9833458430f8b7d8c4a77fd088c'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:13:26.928434 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-9cd033ee-2431-4547-bfe2-915b86266b45 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fa1dc9833458430f8b7d8c4a77fd088c'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:13:26.928434 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-9cd033ee-2431-4547-bfe2-915b86266b45 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fa1dc9833458430f8b7d8c4a77fd088c'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:13:26.939843 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 954/1906] 10.222.0.52 () {58 vars in 1223 bytes} [Wed Jul 13 22:13:26 2022] DELETE /identity/v3/users/fa1dc9833458430f8b7d8c4a77fd088c => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 22:13:27.320786 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ab57e4-9bbe-4fa9-a885-ef0eaafd16e7 admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:13:27.323542 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-60ab57e4-9bbe-4fa9-a885-ef0eaafd16e7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:13:27.340142 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60ab57e4-9bbe-4fa9-a885-ef0eaafd16e7 admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:13:27.344596 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:13:27.401793 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2020898f-ac58-4038-95fe-77ee03735cd9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:13:27.403668 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2020898f-ac58-4038-95fe-77ee03735cd9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:13:27.403964 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2020898f-ac58-4038-95fe-77ee03735cd9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:13:27.404233 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2020898f-ac58-4038-95fe-77ee03735cd9 admin admin] PATH_INFO: `/v3/projects/36a974d17ddb4459b1b48d8b2536d5f5` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:13:27.405456 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2020898f-ac58-4038-95fe-77ee03735cd9 admin admin] RBAC: Authorizing `identity:delete_project(project_id=36a974d17ddb4459b1b48d8b2536d5f5)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:13:27.407467 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2020898f-ac58-4038-95fe-77ee03735cd9 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:13:27.457967 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-2020898f-ac58-4038-95fe-77ee03735cd9 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '36a974d17ddb4459b1b48d8b2536d5f5', 'request_id': 'req-2020898f-ac58-4038-95fe-77ee03735cd9'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:13:27.465021 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-2020898f-ac58-4038-95fe-77ee03735cd9 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '36a974d17ddb4459b1b48d8b2536d5f5', 'request_id': 'req-2020898f-ac58-4038-95fe-77ee03735cd9'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:13:27.472379 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-2020898f-ac58-4038-95fe-77ee03735cd9 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '36a974d17ddb4459b1b48d8b2536d5f5', 'request_id': 'req-2020898f-ac58-4038-95fe-77ee03735cd9'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:13:27.489639 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-2020898f-ac58-4038-95fe-77ee03735cd9 admin admin] The token cache is being invalidate because project 36a974d17ddb4459b1b48d8b2536d5f5 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80382) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 22:13:27.489932 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-2020898f-ac58-4038-95fe-77ee03735cd9 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:13:27.490716 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-2020898f-ac58-4038-95fe-77ee03735cd9 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:13:27.493580 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 953/1907] 10.222.0.52 () {58 vars in 1232 bytes} [Wed Jul 13 22:13:27 2022] DELETE /identity/v3/projects/36a974d17ddb4459b1b48d8b2536d5f5 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 22:13:27.496973 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd033ee-2431-4547-bfe2-915b86266b45 admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:13:27.498633 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-9cd033ee-2431-4547-bfe2-915b86266b45 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:13:27.512128 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cd033ee-2431-4547-bfe2-915b86266b45 admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:13:27.581024 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c1a1e4c6-66a8-4b6c-981e-a2012f4ca231 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:13:27.581989 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a1e4c6-66a8-4b6c-981e-a2012f4ca231 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:13:27.582273 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a1e4c6-66a8-4b6c-981e-a2012f4ca231 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:13:27.582587 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c1a1e4c6-66a8-4b6c-981e-a2012f4ca231 admin admin] PATH_INFO: `/v3/users/c678fa74aac84152afbedfc4b8ba7de9` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:13:27.583811 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a1e4c6-66a8-4b6c-981e-a2012f4ca231 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c678fa74aac84152afbedfc4b8ba7de9)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:13:27.585825 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c1a1e4c6-66a8-4b6c-981e-a2012f4ca231 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:13:27.647667 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-c1a1e4c6-66a8-4b6c-981e-a2012f4ca231 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c678fa74aac84152afbedfc4b8ba7de9'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:13:27.661240 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-c1a1e4c6-66a8-4b6c-981e-a2012f4ca231 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c678fa74aac84152afbedfc4b8ba7de9'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:13:27.674720 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-c1a1e4c6-66a8-4b6c-981e-a2012f4ca231 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c678fa74aac84152afbedfc4b8ba7de9'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:13:27.675506 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-c1a1e4c6-66a8-4b6c-981e-a2012f4ca231 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c678fa74aac84152afbedfc4b8ba7de9'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:13:27.676212 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-c1a1e4c6-66a8-4b6c-981e-a2012f4ca231 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c678fa74aac84152afbedfc4b8ba7de9'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:13:27.688590 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 955/1908] 10.222.0.52 () {58 vars in 1223 bytes} [Wed Jul 13 22:13:27 2022] DELETE /identity/v3/users/c678fa74aac84152afbedfc4b8ba7de9 => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 22:13:28.056331 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2020898f-ac58-4038-95fe-77ee03735cd9 admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:13:28.057807 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-2020898f-ac58-4038-95fe-77ee03735cd9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:13:28.073989 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2020898f-ac58-4038-95fe-77ee03735cd9 admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:13:28.132395 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3cf3243b-4ca8-4a14-8a0b-419e165a2a2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:13:28.133193 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf3243b-4ca8-4a14-8a0b-419e165a2a2c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:13:28.133473 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf3243b-4ca8-4a14-8a0b-419e165a2a2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:13:28.133749 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3cf3243b-4ca8-4a14-8a0b-419e165a2a2c admin admin] PATH_INFO: `/v3/projects/3e340dd42cf540d2ad001b379f71250a` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:13:28.134980 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf3243b-4ca8-4a14-8a0b-419e165a2a2c admin admin] RBAC: Authorizing `identity:delete_project(project_id=3e340dd42cf540d2ad001b379f71250a)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:13:28.136885 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3cf3243b-4ca8-4a14-8a0b-419e165a2a2c admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:13:28.178237 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:13:28.185469 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-3cf3243b-4ca8-4a14-8a0b-419e165a2a2c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3e340dd42cf540d2ad001b379f71250a', 'request_id': 'req-3cf3243b-4ca8-4a14-8a0b-419e165a2a2c'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:13:28.193277 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-3cf3243b-4ca8-4a14-8a0b-419e165a2a2c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3e340dd42cf540d2ad001b379f71250a', 'request_id': 'req-3cf3243b-4ca8-4a14-8a0b-419e165a2a2c'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:13:28.199060 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-3cf3243b-4ca8-4a14-8a0b-419e165a2a2c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3e340dd42cf540d2ad001b379f71250a', 'request_id': 'req-3cf3243b-4ca8-4a14-8a0b-419e165a2a2c'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:13:28.213251 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-3cf3243b-4ca8-4a14-8a0b-419e165a2a2c admin admin] The token cache is being invalidate because project 3e340dd42cf540d2ad001b379f71250a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80382) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 22:13:28.213532 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-3cf3243b-4ca8-4a14-8a0b-419e165a2a2c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:13:28.214218 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-3cf3243b-4ca8-4a14-8a0b-419e165a2a2c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:13:28.215857 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 954/1909] 10.222.0.52 () {58 vars in 1232 bytes} [Wed Jul 13 22:13:28 2022] DELETE /identity/v3/projects/3e340dd42cf540d2ad001b379f71250a => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 22:13:28.221351 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7440bd08-4450-4352-b2bd-d9059626eb1a None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:13:28.221804 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7440bd08-4450-4352-b2bd-d9059626eb1a None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:13:28.222291 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7440bd08-4450-4352-b2bd-d9059626eb1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:13:28.241230 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-7440bd08-4450-4352-b2bd-d9059626eb1a None None] MFA Rules not processed for user `cc99c93da5074cf5a48bc2aa9a92c5e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 22:13:28.242440 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-7440bd08-4450-4352-b2bd-d9059626eb1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:13:28.264467 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=83,keystone:DELETE=15,keystone:INSERT=2 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:13:28.309040 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 956/1910] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jul 13 22:13:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 22:13:28.318693 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e12ae45-8bdd-495b-8bf3-1472a6b1910f None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:13:28.318693 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e12ae45-8bdd-495b-8bf3-1472a6b1910f None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:13:28.318873 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e12ae45-8bdd-495b-8bf3-1472a6b1910f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:13:28.342961 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-5e12ae45-8bdd-495b-8bf3-1472a6b1910f None None] MFA Rules not processed for user `cc99c93da5074cf5a48bc2aa9a92c5e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 22:13:28.344353 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-5e12ae45-8bdd-495b-8bf3-1472a6b1910f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:13:28.355173 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 955/1911] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jul 13 22:13:28 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 38 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 22:13:28.360328 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7440bd08-4450-4352-b2bd-d9059626eb1a None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:13:28.370518 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7440bd08-4450-4352-b2bd-d9059626eb1a None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:13:28.375487 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3a450b-a7bc-45eb-8d89-798b4449ad22 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:13:28.376181 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3a450b-a7bc-45eb-8d89-798b4449ad22 admin admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:13:28.376477 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3a450b-a7bc-45eb-8d89-798b4449ad22 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:13:28.376790 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd3a450b-a7bc-45eb-8d89-798b4449ad22 admin admin] PATH_INFO: `/v3/domains` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:13:28.377608 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd3a450b-a7bc-45eb-8d89-798b4449ad22 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 22:13:28.377961 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd3a450b-a7bc-45eb-8d89-798b4449ad22 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:13:28.379771 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd3a450b-a7bc-45eb-8d89-798b4449ad22 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:13:28.395092 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 957/1912] 10.222.0.52 () {58 vars in 1152 bytes} [Wed Jul 13 22:13:28 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 22:13:28.401667 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e12ae45-8bdd-495b-8bf3-1472a6b1910f None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:13:28.411571 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e12ae45-8bdd-495b-8bf3-1472a6b1910f None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:13:28.416453 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8d984f-8694-497b-bb3f-d1ccabecd37a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:13:28.417148 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8d984f-8694-497b-bb3f-d1ccabecd37a admin admin] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:13:28.417452 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8d984f-8694-497b-bb3f-d1ccabecd37a admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:13:28.417678 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca8d984f-8694-497b-bb3f-d1ccabecd37a admin admin] PATH_INFO: `/v3/projects` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:13:28.418388 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca8d984f-8694-497b-bb3f-d1ccabecd37a admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:13:28.420448 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca8d984f-8694-497b-bb3f-d1ccabecd37a admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:13:28.449047 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 956/1913] 10.222.0.52 () {60 vars in 1152 bytes} [Wed Jul 13 22:13:28 2022] POST /identity/v3/projects => generated 375 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 13 22:13:28.453727 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3a450b-a7bc-45eb-8d89-798b4449ad22 admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:13:28.465434 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd3a450b-a7bc-45eb-8d89-798b4449ad22 admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:13:28.532757 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e632449-6b59-42c5-93c1-b3e6ba142ea5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:13:28.533592 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e632449-6b59-42c5-93c1-b3e6ba142ea5 admin admin] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:13:28.533890 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e632449-6b59-42c5-93c1-b3e6ba142ea5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:13:28.534262 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7e632449-6b59-42c5-93c1-b3e6ba142ea5 admin admin] PATH_INFO: `/v3/users` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:13:28.535180 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e632449-6b59-42c5-93c1-b3e6ba142ea5 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:13:28.537913 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7e632449-6b59-42c5-93c1-b3e6ba142ea5 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:13:28.570618 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 958/1914] 10.222.0.52 () {60 vars in 1143 bytes} [Wed Jul 13 22:13:28 2022] POST /identity/v3/users => generated 288 bytes in 115 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 22:13:28.571821 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8d984f-8694-497b-bb3f-d1ccabecd37a admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:13:28.583743 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca8d984f-8694-497b-bb3f-d1ccabecd37a admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:13:28.589546 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928d935d-6a07-4ae5-b643-ff012d97b1bd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:13:28.590342 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928d935d-6a07-4ae5-b643-ff012d97b1bd admin admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:13:28.590959 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928d935d-6a07-4ae5-b643-ff012d97b1bd admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:13:28.591310 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-928d935d-6a07-4ae5-b643-ff012d97b1bd admin admin] PATH_INFO: `/v3/roles` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:13:28.592119 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928d935d-6a07-4ae5-b643-ff012d97b1bd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:13:28.594013 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-928d935d-6a07-4ae5-b643-ff012d97b1bd admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:13:28.611262 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 957/1915] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jul 13 22:13:28 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 22:13:28.614114 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e632449-6b59-42c5-93c1-b3e6ba142ea5 admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:13:28.626186 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7e632449-6b59-42c5-93c1-b3e6ba142ea5 admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:13:28.635803 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b63d9b09-8458-496c-96bd-228b421ab454 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:13:28.636556 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63d9b09-8458-496c-96bd-228b421ab454 admin admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:13:28.636939 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63d9b09-8458-496c-96bd-228b421ab454 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:13:28.637331 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b63d9b09-8458-496c-96bd-228b421ab454 admin admin] PATH_INFO: `/v3/roles` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:13:28.638213 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63d9b09-8458-496c-96bd-228b421ab454 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:13:28.640231 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b63d9b09-8458-496c-96bd-228b421ab454 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:13:28.655978 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 959/1916] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jul 13 22:13:28 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 22:13:28.656618 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928d935d-6a07-4ae5-b643-ff012d97b1bd admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:13:28.667455 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-928d935d-6a07-4ae5-b643-ff012d97b1bd admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:13:28.672708 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb03a8bc-84f5-43d8-a31e-dc232408aa27 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:13:28.673493 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb03a8bc-84f5-43d8-a31e-dc232408aa27 admin admin] REQUEST_METHOD: `PUT` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:13:28.673769 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb03a8bc-84f5-43d8-a31e-dc232408aa27 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:13:28.674040 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bb03a8bc-84f5-43d8-a31e-dc232408aa27 admin admin] PATH_INFO: `/v3/projects/fcc414b4ee2d4eda9987eec45f38c3b0/users/82419bfe941c480eba0035afc1ee7aa4/roles/2a5e86976394498abb6310697cdb3f8b` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:13:28.690565 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb03a8bc-84f5-43d8-a31e-dc232408aa27 admin admin] RBAC: Authorizing `identity:create_grant(project_id=fcc414b4ee2d4eda9987eec45f38c3b0, user_id=82419bfe941c480eba0035afc1ee7aa4, role_id=2a5e86976394498abb6310697cdb3f8b)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:13:28.693357 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bb03a8bc-84f5-43d8-a31e-dc232408aa27 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:13:28.708503 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 958/1917] 10.222.0.52 () {60 vars in 1482 bytes} [Wed Jul 13 22:13:28 2022] PUT /identity/v3/projects/fcc414b4ee2d4eda9987eec45f38c3b0/users/82419bfe941c480eba0035afc1ee7aa4/roles/2a5e86976394498abb6310697cdb3f8b => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 13 22:13:28.719301 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0908450e-3719-43ca-92ca-f5ef43750613 None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:13:28.719670 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0908450e-3719-43ca-92ca-f5ef43750613 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:13:28.719951 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0908450e-3719-43ca-92ca-f5ef43750613 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:13:28.748929 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-0908450e-3719-43ca-92ca-f5ef43750613 None None] MFA Rules not processed for user `82419bfe941c480eba0035afc1ee7aa4`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 22:13:28.750147 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-0908450e-3719-43ca-92ca-f5ef43750613 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:13:28.819522 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 960/1918] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jul 13 22:13:28 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 22:13:28.831672 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb03a8bc-84f5-43d8-a31e-dc232408aa27 admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:13:28.833844 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-bb03a8bc-84f5-43d8-a31e-dc232408aa27 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:13:28.913233 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bb03a8bc-84f5-43d8-a31e-dc232408aa27 admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:13:28.917032 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7d318ae-d817-4080-863d-8789214d6a34 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc77e1ea1ada4700b809beb1c980c5e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417e3e2f1333481795a089bef2e24bea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:13:28.917700 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d318ae-d817-4080-863d-8789214d6a34 service cinder] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:13:28.917984 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d318ae-d817-4080-863d-8789214d6a34 service cinder] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:13:28.918268 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7d318ae-d817-4080-863d-8789214d6a34 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:13:28.927595 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d318ae-d817-4080-863d-8789214d6a34 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:13:28.929085 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7d318ae-d817-4080-863d-8789214d6a34 service cinder] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:13:28.938425 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 959/1919] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jul 13 22:13:28 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 22:13:38.816149 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=87,keystone:INSERT=3 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:13:38.934221 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=63,keystone:DELETE=4,keystone:INSERT=3 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:13:44.898949 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0908450e-3719-43ca-92ca-f5ef43750613 None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:13:44.900457 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-0908450e-3719-43ca-92ca-f5ef43750613 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:13:44.971335 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0908450e-3719-43ca-92ca-f5ef43750613 None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:13:44.976955 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73fd926d-002b-4d6e-91ec-2fa9b7187903 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:13:44.977861 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fd926d-002b-4d6e-91ec-2fa9b7187903 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:13:44.978136 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fd926d-002b-4d6e-91ec-2fa9b7187903 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:13:44.978407 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73fd926d-002b-4d6e-91ec-2fa9b7187903 admin admin] PATH_INFO: `/v3/users/c0fe2106d5f74363b1d5550f7f523ff9` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:13:44.979757 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73fd926d-002b-4d6e-91ec-2fa9b7187903 admin admin] RBAC: Authorizing `identity:delete_user(user_id=c0fe2106d5f74363b1d5550f7f523ff9)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:13:44.981799 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73fd926d-002b-4d6e-91ec-2fa9b7187903 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:13:45.033340 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-73fd926d-002b-4d6e-91ec-2fa9b7187903 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'c0fe2106d5f74363b1d5550f7f523ff9'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:13:45.041414 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-73fd926d-002b-4d6e-91ec-2fa9b7187903 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'c0fe2106d5f74363b1d5550f7f523ff9'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:13:45.054357 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-73fd926d-002b-4d6e-91ec-2fa9b7187903 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'c0fe2106d5f74363b1d5550f7f523ff9'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:13:45.055051 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-73fd926d-002b-4d6e-91ec-2fa9b7187903 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'c0fe2106d5f74363b1d5550f7f523ff9'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:13:45.055968 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-73fd926d-002b-4d6e-91ec-2fa9b7187903 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'c0fe2106d5f74363b1d5550f7f523ff9'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:13:45.068917 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 961/1920] 10.222.0.52 () {58 vars in 1223 bytes} [Wed Jul 13 22:13:44 2022] DELETE /identity/v3/users/c0fe2106d5f74363b1d5550f7f523ff9 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 22:13:45.074315 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ee0bba-4efe-4fab-9e9d-4c09cf758c7b None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:13:45.074793 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ee0bba-4efe-4fab-9e9d-4c09cf758c7b None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:13:45.074945 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-81ee0bba-4efe-4fab-9e9d-4c09cf758c7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:13:45.094984 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-81ee0bba-4efe-4fab-9e9d-4c09cf758c7b None None] MFA Rules not processed for user `cc99c93da5074cf5a48bc2aa9a92c5e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 22:13:45.096742 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-81ee0bba-4efe-4fab-9e9d-4c09cf758c7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:13:45.158674 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 960/1921] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jul 13 22:13:45 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 13 22:13:45.168977 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73fd926d-002b-4d6e-91ec-2fa9b7187903 admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:13:45.170452 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-73fd926d-002b-4d6e-91ec-2fa9b7187903 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:13:45.231691 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73fd926d-002b-4d6e-91ec-2fa9b7187903 admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:13:45.236064 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e4aebd-35a6-415d-8095-90bfe92b0c40 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '847ec9a1bd4d4dec804318a5c845895a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417e3e2f1333481795a089bef2e24bea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:13:45.236725 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e4aebd-35a6-415d-8095-90bfe92b0c40 service neutron] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:13:45.237103 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e4aebd-35a6-415d-8095-90bfe92b0c40 service neutron] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:13:45.237379 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-51e4aebd-35a6-415d-8095-90bfe92b0c40 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:13:45.247998 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e4aebd-35a6-415d-8095-90bfe92b0c40 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:13:45.249440 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-51e4aebd-35a6-415d-8095-90bfe92b0c40 service neutron] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:13:45.259586 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 962/1922] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jul 13 22:13:45 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 22:13:45.619762 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ee0bba-4efe-4fab-9e9d-4c09cf758c7b None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:13:45.621202 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-81ee0bba-4efe-4fab-9e9d-4c09cf758c7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:13:45.642466 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-81ee0bba-4efe-4fab-9e9d-4c09cf758c7b None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:13:45.649201 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd33361-be14-4f9c-a2a0-0ace9aad1752 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:13:45.650342 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd33361-be14-4f9c-a2a0-0ace9aad1752 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:13:45.651012 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd33361-be14-4f9c-a2a0-0ace9aad1752 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:13:45.651012 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd33361-be14-4f9c-a2a0-0ace9aad1752 admin admin] PATH_INFO: `/v3/projects/ec73f12d9bed4198833dd9a0dad198bb` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:13:45.653774 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd33361-be14-4f9c-a2a0-0ace9aad1752 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ec73f12d9bed4198833dd9a0dad198bb)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:13:45.656024 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bd33361-be14-4f9c-a2a0-0ace9aad1752 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:13:45.724973 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-7bd33361-be14-4f9c-a2a0-0ace9aad1752 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ec73f12d9bed4198833dd9a0dad198bb', 'request_id': 'req-7bd33361-be14-4f9c-a2a0-0ace9aad1752'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:13:45.731751 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-7bd33361-be14-4f9c-a2a0-0ace9aad1752 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ec73f12d9bed4198833dd9a0dad198bb', 'request_id': 'req-7bd33361-be14-4f9c-a2a0-0ace9aad1752'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:13:45.737338 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-7bd33361-be14-4f9c-a2a0-0ace9aad1752 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ec73f12d9bed4198833dd9a0dad198bb', 'request_id': 'req-7bd33361-be14-4f9c-a2a0-0ace9aad1752'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:13:45.751267 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-7bd33361-be14-4f9c-a2a0-0ace9aad1752 admin admin] The token cache is being invalidate because project ec73f12d9bed4198833dd9a0dad198bb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80382) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 22:13:45.751567 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-7bd33361-be14-4f9c-a2a0-0ace9aad1752 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:13:45.752197 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-7bd33361-be14-4f9c-a2a0-0ace9aad1752 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:13:45.754199 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 961/1923] 10.222.0.52 () {58 vars in 1232 bytes} [Wed Jul 13 22:13:45 2022] DELETE /identity/v3/projects/ec73f12d9bed4198833dd9a0dad198bb => generated 0 bytes in 135 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 22:13:45.757971 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e4aebd-35a6-415d-8095-90bfe92b0c40 service neutron] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:13:45.768467 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-51e4aebd-35a6-415d-8095-90bfe92b0c40 service neutron] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:13:45.789225 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:13:45.840015 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3764f360-0da7-490c-8be9-7a9a7cc1dd4f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:13:45.841032 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3764f360-0da7-490c-8be9-7a9a7cc1dd4f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:13:45.841434 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3764f360-0da7-490c-8be9-7a9a7cc1dd4f admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:13:45.841709 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3764f360-0da7-490c-8be9-7a9a7cc1dd4f admin admin] PATH_INFO: `/v3/users/001a0ff2f2844b3c85651a4e1d3dfc46` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:13:45.842963 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3764f360-0da7-490c-8be9-7a9a7cc1dd4f admin admin] RBAC: Authorizing `identity:delete_user(user_id=001a0ff2f2844b3c85651a4e1d3dfc46)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:13:45.845047 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3764f360-0da7-490c-8be9-7a9a7cc1dd4f admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:13:45.897960 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-3764f360-0da7-490c-8be9-7a9a7cc1dd4f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '001a0ff2f2844b3c85651a4e1d3dfc46'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:13:45.908608 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-3764f360-0da7-490c-8be9-7a9a7cc1dd4f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '001a0ff2f2844b3c85651a4e1d3dfc46'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:13:45.923369 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-3764f360-0da7-490c-8be9-7a9a7cc1dd4f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '001a0ff2f2844b3c85651a4e1d3dfc46'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:13:45.924101 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-3764f360-0da7-490c-8be9-7a9a7cc1dd4f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '001a0ff2f2844b3c85651a4e1d3dfc46'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:13:45.925110 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-3764f360-0da7-490c-8be9-7a9a7cc1dd4f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '001a0ff2f2844b3c85651a4e1d3dfc46'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:13:45.937840 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 963/1924] 10.222.0.52 () {58 vars in 1223 bytes} [Wed Jul 13 22:13:45 2022] DELETE /identity/v3/users/001a0ff2f2844b3c85651a4e1d3dfc46 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 22:13:46.220074 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd33361-be14-4f9c-a2a0-0ace9aad1752 admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:13:46.222260 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-7bd33361-be14-4f9c-a2a0-0ace9aad1752 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:13:46.236268 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd33361-be14-4f9c-a2a0-0ace9aad1752 admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:13:46.310198 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44190ce0-b0db-4d0c-9173-f815ec5d4e02 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:13:46.311151 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44190ce0-b0db-4d0c-9173-f815ec5d4e02 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:13:46.311417 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44190ce0-b0db-4d0c-9173-f815ec5d4e02 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:13:46.311691 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-44190ce0-b0db-4d0c-9173-f815ec5d4e02 admin admin] PATH_INFO: `/v3/projects/6f807bd2a3c748e78c50426bf90ff6d9` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:13:46.312909 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44190ce0-b0db-4d0c-9173-f815ec5d4e02 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6f807bd2a3c748e78c50426bf90ff6d9)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:13:46.315167 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-44190ce0-b0db-4d0c-9173-f815ec5d4e02 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:13:46.357246 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:13:46.388502 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-44190ce0-b0db-4d0c-9173-f815ec5d4e02 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6f807bd2a3c748e78c50426bf90ff6d9', 'request_id': 'req-44190ce0-b0db-4d0c-9173-f815ec5d4e02'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:13:46.396615 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-44190ce0-b0db-4d0c-9173-f815ec5d4e02 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6f807bd2a3c748e78c50426bf90ff6d9', 'request_id': 'req-44190ce0-b0db-4d0c-9173-f815ec5d4e02'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:13:46.402151 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-44190ce0-b0db-4d0c-9173-f815ec5d4e02 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6f807bd2a3c748e78c50426bf90ff6d9', 'request_id': 'req-44190ce0-b0db-4d0c-9173-f815ec5d4e02'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:13:46.416026 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-44190ce0-b0db-4d0c-9173-f815ec5d4e02 admin admin] The token cache is being invalidate because project 6f807bd2a3c748e78c50426bf90ff6d9 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80382) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 22:13:46.416314 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-44190ce0-b0db-4d0c-9173-f815ec5d4e02 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:13:46.416942 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-44190ce0-b0db-4d0c-9173-f815ec5d4e02 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:13:46.418578 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 962/1925] 10.222.0.52 () {58 vars in 1232 bytes} [Wed Jul 13 22:13:46 2022] DELETE /identity/v3/projects/6f807bd2a3c748e78c50426bf90ff6d9 => generated 0 bytes in 199 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 22:13:46.423510 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f8c7ed-6abe-4887-8c1b-e72272d8ae1a None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:13:46.423971 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f8c7ed-6abe-4887-8c1b-e72272d8ae1a None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:13:46.424244 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6f8c7ed-6abe-4887-8c1b-e72272d8ae1a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:13:46.442969 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-a6f8c7ed-6abe-4887-8c1b-e72272d8ae1a None None] MFA Rules not processed for user `cc99c93da5074cf5a48bc2aa9a92c5e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 22:13:46.444213 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-a6f8c7ed-6abe-4887-8c1b-e72272d8ae1a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:13:46.508060 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 964/1926] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jul 13 22:13:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 22:13:46.513781 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44190ce0-b0db-4d0c-9173-f815ec5d4e02 admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:13:46.515592 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-44190ce0-b0db-4d0c-9173-f815ec5d4e02 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:13:46.517022 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99eb25fd-66bc-483e-9b58-bdb50b0d6795 None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:13:46.517552 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99eb25fd-66bc-483e-9b58-bdb50b0d6795 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:13:46.518241 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99eb25fd-66bc-483e-9b58-bdb50b0d6795 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:13:46.530189 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-44190ce0-b0db-4d0c-9173-f815ec5d4e02 admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:13:46.535732 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5d8c95be-a5c4-4b01-82ab-90728c74c9ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:13:46.536822 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8c95be-a5c4-4b01-82ab-90728c74c9ef admin admin] REQUEST_METHOD: `DELETE` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:13:46.537206 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8c95be-a5c4-4b01-82ab-90728c74c9ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:13:46.537581 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5d8c95be-a5c4-4b01-82ab-90728c74c9ef admin admin] PATH_INFO: `/v3/users/82419bfe941c480eba0035afc1ee7aa4` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:13:46.539125 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8c95be-a5c4-4b01-82ab-90728c74c9ef admin admin] RBAC: Authorizing `identity:delete_user(user_id=82419bfe941c480eba0035afc1ee7aa4)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:13:46.541202 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5d8c95be-a5c4-4b01-82ab-90728c74c9ef admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:13:46.553016 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-99eb25fd-66bc-483e-9b58-bdb50b0d6795 None None] MFA Rules not processed for user `cc99c93da5074cf5a48bc2aa9a92c5e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 22:13:46.554234 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-99eb25fd-66bc-483e-9b58-bdb50b0d6795 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:13:46.565116 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 965/1927] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jul 13 22:13:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 49 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 22:13:46.569172 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99eb25fd-66bc-483e-9b58-bdb50b0d6795 None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:13:46.582270 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99eb25fd-66bc-483e-9b58-bdb50b0d6795 None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:13:46.587911 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c157780-8adc-4a6d-8108-954d785462c7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:13:46.588638 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c157780-8adc-4a6d-8108-954d785462c7 admin admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:13:46.588911 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c157780-8adc-4a6d-8108-954d785462c7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:13:46.589181 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c157780-8adc-4a6d-8108-954d785462c7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:13:46.589954 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c157780-8adc-4a6d-8108-954d785462c7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 22:13:46.590215 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c157780-8adc-4a6d-8108-954d785462c7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:13:46.591924 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c157780-8adc-4a6d-8108-954d785462c7 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:13:46.608452 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 966/1928] 10.222.0.52 () {58 vars in 1152 bytes} [Wed Jul 13 22:13:46 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 22:13:46.609448 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c157780-8adc-4a6d-8108-954d785462c7 admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:13:46.620777 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-5d8c95be-a5c4-4b01-82ab-90728c74c9ef admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '82419bfe941c480eba0035afc1ee7aa4'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:13:46.621714 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c157780-8adc-4a6d-8108-954d785462c7 admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:13:46.629448 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19812af0-d83a-48ac-854f-05645a516763 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:13:46.630178 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19812af0-d83a-48ac-854f-05645a516763 admin admin] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:13:46.630473 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19812af0-d83a-48ac-854f-05645a516763 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:13:46.630810 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-19812af0-d83a-48ac-854f-05645a516763 admin admin] PATH_INFO: `/v3/projects` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:13:46.631467 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19812af0-d83a-48ac-854f-05645a516763 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:13:46.633454 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-19812af0-d83a-48ac-854f-05645a516763 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:13:46.635819 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-5d8c95be-a5c4-4b01-82ab-90728c74c9ef admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '82419bfe941c480eba0035afc1ee7aa4'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:13:46.653911 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-5d8c95be-a5c4-4b01-82ab-90728c74c9ef admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '82419bfe941c480eba0035afc1ee7aa4'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:13:46.655130 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-5d8c95be-a5c4-4b01-82ab-90728c74c9ef admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '82419bfe941c480eba0035afc1ee7aa4'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:13:46.656137 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-5d8c95be-a5c4-4b01-82ab-90728c74c9ef admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '82419bfe941c480eba0035afc1ee7aa4'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:13:46.658210 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:13:46.666548 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 967/1929] 10.222.0.52 () {60 vars in 1152 bytes} [Wed Jul 13 22:13:46 2022] POST /identity/v3/projects => generated 387 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 13 22:13:46.671411 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19812af0-d83a-48ac-854f-05645a516763 admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:13:46.673860 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-19812af0-d83a-48ac-854f-05645a516763 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:13:46.676365 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 963/1930] 10.222.0.52 () {58 vars in 1223 bytes} [Wed Jul 13 22:13:46 2022] DELETE /identity/v3/users/82419bfe941c480eba0035afc1ee7aa4 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 22:13:46.683358 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819360f4-bf06-4e9d-9366-891bef394157 None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:13:46.683640 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819360f4-bf06-4e9d-9366-891bef394157 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:13:46.683950 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-819360f4-bf06-4e9d-9366-891bef394157 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:13:46.712518 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-819360f4-bf06-4e9d-9366-891bef394157 None None] MFA Rules not processed for user `cc99c93da5074cf5a48bc2aa9a92c5e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 22:13:46.714698 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-819360f4-bf06-4e9d-9366-891bef394157 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:13:46.790514 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-19812af0-d83a-48ac-854f-05645a516763 admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:13:46.882884 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 964/1931] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jul 13 22:13:46 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 201 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 22:13:46.906139 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6fe8ccc-a0fa-4202-8a25-a58920c643b1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:13:46.907453 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819360f4-bf06-4e9d-9366-891bef394157 None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:13:46.908025 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fe8ccc-a0fa-4202-8a25-a58920c643b1 admin admin] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:13:46.908652 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fe8ccc-a0fa-4202-8a25-a58920c643b1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:13:46.911978 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6fe8ccc-a0fa-4202-8a25-a58920c643b1 admin admin] PATH_INFO: `/v3/users` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:13:46.913536 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-819360f4-bf06-4e9d-9366-891bef394157 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:13:46.914069 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6fe8ccc-a0fa-4202-8a25-a58920c643b1 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:13:46.918834 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6fe8ccc-a0fa-4202-8a25-a58920c643b1 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:13:46.966867 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:DELETE=16,keystone:SELECT=82,keystone:INSERT=2 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:13:46.978482 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 968/1932] 10.222.0.52 () {60 vars in 1143 bytes} [Wed Jul 13 22:13:46 2022] POST /identity/v3/users => generated 294 bytes in 307 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 22:13:46.983447 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6fe8ccc-a0fa-4202-8a25-a58920c643b1 admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:13:46.999705 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6fe8ccc-a0fa-4202-8a25-a58920c643b1 admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:13:47.005012 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e459d173-2162-493f-92a0-93156708e3d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:13:47.006064 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e459d173-2162-493f-92a0-93156708e3d2 admin admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:13:47.006212 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e459d173-2162-493f-92a0-93156708e3d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:13:47.006506 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e459d173-2162-493f-92a0-93156708e3d2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:13:47.007350 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e459d173-2162-493f-92a0-93156708e3d2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:13:47.008938 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e459d173-2162-493f-92a0-93156708e3d2 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:13:47.021049 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 969/1933] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jul 13 22:13:46 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 22:13:47.026275 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e459d173-2162-493f-92a0-93156708e3d2 admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:13:47.031411 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-819360f4-bf06-4e9d-9366-891bef394157 None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:13:47.039691 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c675ca-4e27-4964-b380-24bdba9dfbdc service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '847ec9a1bd4d4dec804318a5c845895a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417e3e2f1333481795a089bef2e24bea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:13:47.040571 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c675ca-4e27-4964-b380-24bdba9dfbdc service neutron] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:13:47.041125 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c675ca-4e27-4964-b380-24bdba9dfbdc service neutron] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:13:47.041495 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08c675ca-4e27-4964-b380-24bdba9dfbdc service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:13:47.046783 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e459d173-2162-493f-92a0-93156708e3d2 admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:13:47.054986 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c675ca-4e27-4964-b380-24bdba9dfbdc service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:13:47.056765 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08c675ca-4e27-4964-b380-24bdba9dfbdc service neutron] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:13:47.057224 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb42f69f-9667-430f-b937-693c46bbd74e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:13:47.057962 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb42f69f-9667-430f-b937-693c46bbd74e admin admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:13:47.059141 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb42f69f-9667-430f-b937-693c46bbd74e admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:13:47.059141 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb42f69f-9667-430f-b937-693c46bbd74e admin admin] PATH_INFO: `/v3/roles` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:13:47.059464 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb42f69f-9667-430f-b937-693c46bbd74e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:13:47.061698 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb42f69f-9667-430f-b937-693c46bbd74e admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:13:47.069034 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 965/1934] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jul 13 22:13:46 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 164 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 22:13:47.072955 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 970/1935] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jul 13 22:13:47 2022] GET /identity/v3/roles => generated 1420 bytes in 47 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 22:13:47.077168 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c675ca-4e27-4964-b380-24bdba9dfbdc service neutron] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:13:47.088083 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08c675ca-4e27-4964-b380-24bdba9dfbdc service neutron] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:13:47.093497 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae8efd1-b9f8-47f6-96e5-12c01a2a56c8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:13:47.094533 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae8efd1-b9f8-47f6-96e5-12c01a2a56c8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:13:47.094805 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae8efd1-b9f8-47f6-96e5-12c01a2a56c8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:13:47.095080 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0ae8efd1-b9f8-47f6-96e5-12c01a2a56c8 admin admin] PATH_INFO: `/v3/projects/ccfa084ce0f14f92b7458b4e21df297c/users/04a89b54757c4dceb7ee492c9fa6ae67/roles/2a5e86976394498abb6310697cdb3f8b` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:13:47.113031 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae8efd1-b9f8-47f6-96e5-12c01a2a56c8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ccfa084ce0f14f92b7458b4e21df297c, user_id=04a89b54757c4dceb7ee492c9fa6ae67, role_id=2a5e86976394498abb6310697cdb3f8b)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:13:47.115876 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0ae8efd1-b9f8-47f6-96e5-12c01a2a56c8 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:13:47.132255 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 966/1936] 10.222.0.52 () {60 vars in 1482 bytes} [Wed Jul 13 22:13:47 2022] PUT /identity/v3/projects/ccfa084ce0f14f92b7458b4e21df297c/users/04a89b54757c4dceb7ee492c9fa6ae67/roles/2a5e86976394498abb6310697cdb3f8b => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 22:13:47.142960 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec466c5e-d692-4456-bb26-19feda0fc0bf None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:13:47.143273 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec466c5e-d692-4456-bb26-19feda0fc0bf None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:13:47.143548 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ec466c5e-d692-4456-bb26-19feda0fc0bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:13:47.173643 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-ec466c5e-d692-4456-bb26-19feda0fc0bf None None] MFA Rules not processed for user `04a89b54757c4dceb7ee492c9fa6ae67`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 22:13:47.174967 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-ec466c5e-d692-4456-bb26-19feda0fc0bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:13:47.267353 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 971/1937] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jul 13 22:13:47 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 125 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 22:13:47.272821 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae8efd1-b9f8-47f6-96e5-12c01a2a56c8 admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:13:47.363359 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0ae8efd1-b9f8-47f6-96e5-12c01a2a56c8 admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:13:47.368722 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e7bda1-8084-4941-8a91-fc2d2536aa49 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:13:47.369542 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e7bda1-8084-4941-8a91-fc2d2536aa49 admin admin] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:13:47.369805 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e7bda1-8084-4941-8a91-fc2d2536aa49 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:13:47.370031 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-94e7bda1-8084-4941-8a91-fc2d2536aa49 admin admin] PATH_INFO: `/v3/projects` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:13:47.370763 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e7bda1-8084-4941-8a91-fc2d2536aa49 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:13:47.372714 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-94e7bda1-8084-4941-8a91-fc2d2536aa49 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:13:47.405257 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 967/1938] 10.222.0.52 () {60 vars in 1152 bytes} [Wed Jul 13 22:13:47 2022] POST /identity/v3/projects => generated 387 bytes in 133 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 13 22:13:47.410634 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec466c5e-d692-4456-bb26-19feda0fc0bf None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:13:47.417858 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:13:47.422591 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ec466c5e-d692-4456-bb26-19feda0fc0bf None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:13:47.460966 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e7bda1-8084-4941-8a91-fc2d2536aa49 admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:13:47.463422 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-94e7bda1-8084-4941-8a91-fc2d2536aa49 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:13:47.498439 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5863c501-3859-49e4-a745-ff067bb66b8f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:13:47.498789 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5863c501-3859-49e4-a745-ff067bb66b8f admin admin] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:13:47.499309 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5863c501-3859-49e4-a745-ff067bb66b8f admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:13:47.499669 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5863c501-3859-49e4-a745-ff067bb66b8f admin admin] PATH_INFO: `/v3/users` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:13:47.501007 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5863c501-3859-49e4-a745-ff067bb66b8f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:13:47.503757 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5863c501-3859-49e4-a745-ff067bb66b8f admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:13:47.540232 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 972/1939] 10.222.0.52 () {60 vars in 1143 bytes} [Wed Jul 13 22:13:47 2022] POST /identity/v3/users => generated 300 bytes in 131 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 13 22:13:47.541270 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5863c501-3859-49e4-a745-ff067bb66b8f admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:13:47.555728 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5863c501-3859-49e4-a745-ff067bb66b8f admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:13:47.561299 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-94e7bda1-8084-4941-8a91-fc2d2536aa49 admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:13:47.564791 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73553d08-d094-47d8-a16e-e254541a0016 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:13:47.565589 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73553d08-d094-47d8-a16e-e254541a0016 admin admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:13:47.565871 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73553d08-d094-47d8-a16e-e254541a0016 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:13:47.566141 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-73553d08-d094-47d8-a16e-e254541a0016 admin admin] PATH_INFO: `/v3/roles` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:13:47.566916 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73553d08-d094-47d8-a16e-e254541a0016 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:13:47.567036 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db4940d0-d063-4a6f-a353-70a291b56569 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:13:47.567851 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4940d0-d063-4a6f-a353-70a291b56569 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:13:47.568129 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4940d0-d063-4a6f-a353-70a291b56569 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:13:47.568454 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db4940d0-d063-4a6f-a353-70a291b56569 admin admin] PATH_INFO: `/v3/projects/fcc414b4ee2d4eda9987eec45f38c3b0` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:13:47.569685 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4940d0-d063-4a6f-a353-70a291b56569 admin admin] RBAC: Authorizing `identity:delete_project(project_id=fcc414b4ee2d4eda9987eec45f38c3b0)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:13:47.571673 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db4940d0-d063-4a6f-a353-70a291b56569 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:13:47.573069 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-73553d08-d094-47d8-a16e-e254541a0016 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:13:47.591793 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 973/1940] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jul 13 22:13:47 2022] GET /identity/v3/roles => generated 1420 bytes in 51 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 22:13:47.593046 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73553d08-d094-47d8-a16e-e254541a0016 admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:13:47.604526 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-73553d08-d094-47d8-a16e-e254541a0016 admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:13:47.611589 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c443a7c6-237c-4c03-9af1-f5ba9261dcb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:13:47.612542 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c443a7c6-237c-4c03-9af1-f5ba9261dcb3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:13:47.612821 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c443a7c6-237c-4c03-9af1-f5ba9261dcb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:13:47.613731 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c443a7c6-237c-4c03-9af1-f5ba9261dcb3 admin admin] PATH_INFO: `/v3/projects/211150b302aa41449f963175857b9c9f/users/8cdb18111a6d462ab779ad057c11c122/roles/44efba2579f342df8a0789c59d163ec3` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:13:47.624099 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=96,keystone:INSERT=2,keystone:DELETE=2 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:13:47.650791 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-db4940d0-d063-4a6f-a353-70a291b56569 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'fcc414b4ee2d4eda9987eec45f38c3b0', 'request_id': 'req-db4940d0-d063-4a6f-a353-70a291b56569'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:13:47.651634 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c443a7c6-237c-4c03-9af1-f5ba9261dcb3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=211150b302aa41449f963175857b9c9f, user_id=8cdb18111a6d462ab779ad057c11c122, role_id=44efba2579f342df8a0789c59d163ec3)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:13:47.655638 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c443a7c6-237c-4c03-9af1-f5ba9261dcb3 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:13:47.657578 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-db4940d0-d063-4a6f-a353-70a291b56569 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'fcc414b4ee2d4eda9987eec45f38c3b0', 'request_id': 'req-db4940d0-d063-4a6f-a353-70a291b56569'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:13:47.664218 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-db4940d0-d063-4a6f-a353-70a291b56569 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'fcc414b4ee2d4eda9987eec45f38c3b0', 'request_id': 'req-db4940d0-d063-4a6f-a353-70a291b56569'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:13:47.673786 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 974/1941] 10.222.0.52 () {60 vars in 1482 bytes} [Wed Jul 13 22:13:47 2022] PUT /identity/v3/projects/211150b302aa41449f963175857b9c9f/users/8cdb18111a6d462ab779ad057c11c122/roles/44efba2579f342df8a0789c59d163ec3 => generated 0 bytes in 81 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 22:13:47.677975 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c443a7c6-237c-4c03-9af1-f5ba9261dcb3 admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:13:47.679212 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-db4940d0-d063-4a6f-a353-70a291b56569 admin admin] The token cache is being invalidate because project fcc414b4ee2d4eda9987eec45f38c3b0 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80382) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 22:13:47.679505 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-db4940d0-d063-4a6f-a353-70a291b56569 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:13:47.680226 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-db4940d0-d063-4a6f-a353-70a291b56569 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:13:47.682547 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 968/1942] 10.222.0.52 () {58 vars in 1232 bytes} [Wed Jul 13 22:13:47 2022] DELETE /identity/v3/projects/fcc414b4ee2d4eda9987eec45f38c3b0 => generated 0 bytes in 222 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 13 22:13:47.687334 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9d67b1-d6a9-4ec3-a75c-da6ea85d9c07 None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:13:47.687945 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9d67b1-d6a9-4ec3-a75c-da6ea85d9c07 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:13:47.688451 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a9d67b1-d6a9-4ec3-a75c-da6ea85d9c07 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:13:47.711634 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-6a9d67b1-d6a9-4ec3-a75c-da6ea85d9c07 None None] MFA Rules not processed for user `cc99c93da5074cf5a48bc2aa9a92c5e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 22:13:47.713270 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-6a9d67b1-d6a9-4ec3-a75c-da6ea85d9c07 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:13:47.750359 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c443a7c6-237c-4c03-9af1-f5ba9261dcb3 admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:13:47.755712 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b744e5-8ab2-46bb-a5ac-f31643590d3f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:13:47.756400 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b744e5-8ab2-46bb-a5ac-f31643590d3f admin admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:13:47.756684 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b744e5-8ab2-46bb-a5ac-f31643590d3f admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:13:47.756958 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-22b744e5-8ab2-46bb-a5ac-f31643590d3f admin admin] PATH_INFO: `/v3/roles` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:13:47.757698 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b744e5-8ab2-46bb-a5ac-f31643590d3f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:13:47.759424 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-22b744e5-8ab2-46bb-a5ac-f31643590d3f admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:13:47.771057 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 975/1943] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jul 13 22:13:47 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 13 22:13:47.775098 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b744e5-8ab2-46bb-a5ac-f31643590d3f admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:13:47.776776 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-22b744e5-8ab2-46bb-a5ac-f31643590d3f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:13:47.781062 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 969/1944] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jul 13 22:13:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 13 22:13:47.789228 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822c6e84-8c6b-418e-ab8d-a2046ff549f4 None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:13:47.789656 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822c6e84-8c6b-418e-ab8d-a2046ff549f4 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:13:47.789937 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-822c6e84-8c6b-418e-ab8d-a2046ff549f4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:13:47.790028 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-22b744e5-8ab2-46bb-a5ac-f31643590d3f admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:13:47.795003 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13a6168d-f7c6-40d6-91d4-ac56fc77673c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:13:47.795893 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a6168d-f7c6-40d6-91d4-ac56fc77673c admin admin] REQUEST_METHOD: `PUT` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:13:47.796175 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a6168d-f7c6-40d6-91d4-ac56fc77673c admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:13:47.796478 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13a6168d-f7c6-40d6-91d4-ac56fc77673c admin admin] PATH_INFO: `/v3/projects/211150b302aa41449f963175857b9c9f/users/8cdb18111a6d462ab779ad057c11c122/roles/44efba2579f342df8a0789c59d163ec3` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:13:47.798516 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a6168d-f7c6-40d6-91d4-ac56fc77673c admin admin] RBAC: Authorizing `identity:create_grant(project_id=211150b302aa41449f963175857b9c9f, user_id=8cdb18111a6d462ab779ad057c11c122, role_id=44efba2579f342df8a0789c59d163ec3)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:13:47.801267 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13a6168d-f7c6-40d6-91d4-ac56fc77673c admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:13:47.807991 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-822c6e84-8c6b-418e-ab8d-a2046ff549f4 None None] MFA Rules not processed for user `cc99c93da5074cf5a48bc2aa9a92c5e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 22:13:47.809189 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-822c6e84-8c6b-418e-ab8d-a2046ff549f4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:13:47.811836 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 976/1945] 10.222.0.52 () {60 vars in 1482 bytes} [Wed Jul 13 22:13:47 2022] PUT /identity/v3/projects/211150b302aa41449f963175857b9c9f/users/8cdb18111a6d462ab779ad057c11c122/roles/44efba2579f342df8a0789c59d163ec3 => generated 0 bytes in 37 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 22:13:47.823753 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4a1111-56ef-40a1-a41b-5a1c06f64e6a None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:13:47.823962 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4a1111-56ef-40a1-a41b-5a1c06f64e6a None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:13:47.824242 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d4a1111-56ef-40a1-a41b-5a1c06f64e6a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:13:47.851222 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-9d4a1111-56ef-40a1-a41b-5a1c06f64e6a None None] MFA Rules not processed for user `8cdb18111a6d462ab779ad057c11c122`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 22:13:47.852458 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-9d4a1111-56ef-40a1-a41b-5a1c06f64e6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:13:47.869887 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 970/1946] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jul 13 22:13:47 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 13 22:13:47.874017 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822c6e84-8c6b-418e-ab8d-a2046ff549f4 None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:13:47.883083 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-822c6e84-8c6b-418e-ab8d-a2046ff549f4 None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:13:47.884433 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:13:47.888043 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00da390-79d4-47f6-a1a1-ebe1b7db4c5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:13:47.888710 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00da390-79d4-47f6-a1a1-ebe1b7db4c5c admin admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:13:47.888982 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00da390-79d4-47f6-a1a1-ebe1b7db4c5c admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:13:47.889249 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a00da390-79d4-47f6-a1a1-ebe1b7db4c5c admin admin] PATH_INFO: `/v3/domains` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:13:47.889977 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a00da390-79d4-47f6-a1a1-ebe1b7db4c5c admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 22:13:47.890278 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a00da390-79d4-47f6-a1a1-ebe1b7db4c5c admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:13:47.891932 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a00da390-79d4-47f6-a1a1-ebe1b7db4c5c admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:13:47.903833 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 971/1947] 10.222.0.52 () {58 vars in 1152 bytes} [Wed Jul 13 22:13:47 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 22:13:47.908028 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00da390-79d4-47f6-a1a1-ebe1b7db4c5c admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:13:47.917929 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a00da390-79d4-47f6-a1a1-ebe1b7db4c5c admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:13:47.922832 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5d5767-ea6c-4545-be32-1fb4db6716da admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:13:47.923464 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5d5767-ea6c-4545-be32-1fb4db6716da admin admin] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:13:47.923729 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5d5767-ea6c-4545-be32-1fb4db6716da admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:13:47.924001 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf5d5767-ea6c-4545-be32-1fb4db6716da admin admin] PATH_INFO: `/v3/projects` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:13:47.924708 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf5d5767-ea6c-4545-be32-1fb4db6716da admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:13:47.926567 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 977/1948] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jul 13 22:13:47 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 13 22:13:47.930368 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bf5d5767-ea6c-4545-be32-1fb4db6716da admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:13:47.937803 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4a1111-56ef-40a1-a41b-5a1c06f64e6a None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:13:47.939186 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-9d4a1111-56ef-40a1-a41b-5a1c06f64e6a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:13:47.958350 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 972/1949] 10.222.0.52 () {60 vars in 1152 bytes} [Wed Jul 13 22:13:47 2022] POST /identity/v3/projects => generated 399 bytes in 51 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 13 22:13:47.962325 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5d5767-ea6c-4545-be32-1fb4db6716da admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:13:47.991926 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=93,keystone:DELETE=5,keystone:INSERT=2 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:13:48.001682 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d4a1111-56ef-40a1-a41b-5a1c06f64e6a None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:13:48.025969 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf5d5767-ea6c-4545-be32-1fb4db6716da admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:13:48.031059 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85fb8e71-a37f-4350-b80d-87e0f3f47436 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:13:48.031738 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fb8e71-a37f-4350-b80d-87e0f3f47436 admin admin] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:13:48.032020 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fb8e71-a37f-4350-b80d-87e0f3f47436 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:13:48.032294 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-85fb8e71-a37f-4350-b80d-87e0f3f47436 admin admin] PATH_INFO: `/v3/users` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:13:48.032996 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85fb8e71-a37f-4350-b80d-87e0f3f47436 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:13:48.035060 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-85fb8e71-a37f-4350-b80d-87e0f3f47436 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:13:48.052383 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd5faf4-76f3-4b36-8232-5617bf58e0de service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc77e1ea1ada4700b809beb1c980c5e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417e3e2f1333481795a089bef2e24bea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:13:48.053097 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd5faf4-76f3-4b36-8232-5617bf58e0de service cinder] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:13:48.053404 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd5faf4-76f3-4b36-8232-5617bf58e0de service cinder] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:13:48.053712 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffd5faf4-76f3-4b36-8232-5617bf58e0de service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:13:48.061020 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 973/1950] 10.222.0.52 () {60 vars in 1143 bytes} [Wed Jul 13 22:13:47 2022] POST /identity/v3/users => generated 300 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 13 22:13:48.065148 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85fb8e71-a37f-4350-b80d-87e0f3f47436 admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:13:48.077601 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-85fb8e71-a37f-4350-b80d-87e0f3f47436 admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:13:48.082480 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5641adb4-a909-4aea-a6cb-28de66cb571e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:13:48.083158 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5641adb4-a909-4aea-a6cb-28de66cb571e admin admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:13:48.083443 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5641adb4-a909-4aea-a6cb-28de66cb571e admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:13:48.083726 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5641adb4-a909-4aea-a6cb-28de66cb571e admin admin] PATH_INFO: `/v3/roles` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:13:48.084587 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5641adb4-a909-4aea-a6cb-28de66cb571e admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:13:48.086211 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5641adb4-a909-4aea-a6cb-28de66cb571e admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:13:48.100174 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 974/1951] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jul 13 22:13:48 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 22:13:48.101867 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5641adb4-a909-4aea-a6cb-28de66cb571e admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:13:48.115538 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5641adb4-a909-4aea-a6cb-28de66cb571e admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:13:48.121538 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba22e0cb-bcb7-4c71-8735-c709a43efafd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:13:48.122377 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba22e0cb-bcb7-4c71-8735-c709a43efafd admin admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:13:48.122545 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba22e0cb-bcb7-4c71-8735-c709a43efafd admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:13:48.122776 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba22e0cb-bcb7-4c71-8735-c709a43efafd admin admin] PATH_INFO: `/v3/roles` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:13:48.123606 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba22e0cb-bcb7-4c71-8735-c709a43efafd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:13:48.125062 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd5faf4-76f3-4b36-8232-5617bf58e0de service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:13:48.125562 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba22e0cb-bcb7-4c71-8735-c709a43efafd admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:13:48.126528 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffd5faf4-76f3-4b36-8232-5617bf58e0de service cinder] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:13:48.137610 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 978/1952] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jul 13 22:13:47 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 200 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 13 22:13:48.138072 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 975/1953] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jul 13 22:13:48 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 22:13:48.142175 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd5faf4-76f3-4b36-8232-5617bf58e0de service cinder] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:13:48.158327 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffd5faf4-76f3-4b36-8232-5617bf58e0de service cinder] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:13:48.168358 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18376dbf-c42c-406b-aa58-09105a2f7e48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:13:48.168358 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18376dbf-c42c-406b-aa58-09105a2f7e48 admin admin] REQUEST_METHOD: `PUT` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:13:48.168358 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18376dbf-c42c-406b-aa58-09105a2f7e48 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:13:48.168358 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18376dbf-c42c-406b-aa58-09105a2f7e48 admin admin] PATH_INFO: `/v3/projects/f95251554aaf49e39c01d1b367074fcf/users/1bbc07a53d5d48af921e0aebe00d6da7/roles/2a5e86976394498abb6310697cdb3f8b` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:13:48.169599 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba22e0cb-bcb7-4c71-8735-c709a43efafd admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:13:48.181655 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba22e0cb-bcb7-4c71-8735-c709a43efafd admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:13:48.188006 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a43f2835-190d-4b4e-865b-0b907ccd8f85 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:13:48.188908 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43f2835-190d-4b4e-865b-0b907ccd8f85 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:13:48.189192 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43f2835-190d-4b4e-865b-0b907ccd8f85 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:13:48.189467 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a43f2835-190d-4b4e-865b-0b907ccd8f85 admin admin] PATH_INFO: `/v3/users/04a89b54757c4dceb7ee492c9fa6ae67` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:13:48.190864 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a43f2835-190d-4b4e-865b-0b907ccd8f85 admin admin] RBAC: Authorizing `identity:delete_user(user_id=04a89b54757c4dceb7ee492c9fa6ae67)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:13:48.191311 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18376dbf-c42c-406b-aa58-09105a2f7e48 admin admin] RBAC: Authorizing `identity:create_grant(project_id=f95251554aaf49e39c01d1b367074fcf, user_id=1bbc07a53d5d48af921e0aebe00d6da7, role_id=2a5e86976394498abb6310697cdb3f8b)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:13:48.192654 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a43f2835-190d-4b4e-865b-0b907ccd8f85 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:13:48.194066 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18376dbf-c42c-406b-aa58-09105a2f7e48 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:13:48.212319 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 979/1954] 10.222.0.52 () {60 vars in 1482 bytes} [Wed Jul 13 22:13:48 2022] PUT /identity/v3/projects/f95251554aaf49e39c01d1b367074fcf/users/1bbc07a53d5d48af921e0aebe00d6da7/roles/2a5e86976394498abb6310697cdb3f8b => generated 0 bytes in 70 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 22:13:48.258083 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-a43f2835-190d-4b4e-865b-0b907ccd8f85 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '04a89b54757c4dceb7ee492c9fa6ae67'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:13:48.270027 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-a43f2835-190d-4b4e-865b-0b907ccd8f85 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '04a89b54757c4dceb7ee492c9fa6ae67'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:13:48.285978 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15996be4-4567-4b6f-bb9e-2a4f91f6832e None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:13:48.286337 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15996be4-4567-4b6f-bb9e-2a4f91f6832e None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:13:48.286718 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-15996be4-4567-4b6f-bb9e-2a4f91f6832e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:13:48.290206 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-a43f2835-190d-4b4e-865b-0b907ccd8f85 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '04a89b54757c4dceb7ee492c9fa6ae67'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:13:48.291740 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-a43f2835-190d-4b4e-865b-0b907ccd8f85 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '04a89b54757c4dceb7ee492c9fa6ae67'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:13:48.292503 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-a43f2835-190d-4b4e-865b-0b907ccd8f85 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '04a89b54757c4dceb7ee492c9fa6ae67'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:13:48.308196 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 976/1955] 10.222.0.52 () {58 vars in 1223 bytes} [Wed Jul 13 22:13:48 2022] DELETE /identity/v3/users/04a89b54757c4dceb7ee492c9fa6ae67 => generated 0 bytes in 139 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 13 22:13:48.309836 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:13:48.313359 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401f6762-f842-4359-9363-650e5f156d1f None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:13:48.313718 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401f6762-f842-4359-9363-650e5f156d1f None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:13:48.315241 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-401f6762-f842-4359-9363-650e5f156d1f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:13:48.321599 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-15996be4-4567-4b6f-bb9e-2a4f91f6832e None None] MFA Rules not processed for user `1bbc07a53d5d48af921e0aebe00d6da7`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 22:13:48.323880 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-15996be4-4567-4b6f-bb9e-2a4f91f6832e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:13:48.337430 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-401f6762-f842-4359-9363-650e5f156d1f None None] MFA Rules not processed for user `cc99c93da5074cf5a48bc2aa9a92c5e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 22:13:48.339707 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-401f6762-f842-4359-9363-650e5f156d1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:13:48.395234 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 980/1956] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jul 13 22:13:48 2022] POST /identity/v3/auth/tokens => generated 3266 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 13 22:13:48.407140 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15996be4-4567-4b6f-bb9e-2a4f91f6832e None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:13:48.408490 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-15996be4-4567-4b6f-bb9e-2a4f91f6832e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:13:48.410411 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 977/1957] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jul 13 22:13:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 22:13:48.420350 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401f6762-f842-4359-9363-650e5f156d1f None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:13:48.421715 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-401f6762-f842-4359-9363-650e5f156d1f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:13:48.432039 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=86,keystone:INSERT=4,keystone:DELETE=10 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:13:48.470210 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-15996be4-4567-4b6f-bb9e-2a4f91f6832e None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:13:48.474006 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7713e05a-f2ac-458a-b6ea-fb8e30a688fa service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc77e1ea1ada4700b809beb1c980c5e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417e3e2f1333481795a089bef2e24bea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:13:48.474694 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7713e05a-f2ac-458a-b6ea-fb8e30a688fa service cinder] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:13:48.475089 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7713e05a-f2ac-458a-b6ea-fb8e30a688fa service cinder] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:13:48.475372 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7713e05a-f2ac-458a-b6ea-fb8e30a688fa service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:13:48.484313 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7713e05a-f2ac-458a-b6ea-fb8e30a688fa service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:13:48.486022 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7713e05a-f2ac-458a-b6ea-fb8e30a688fa service cinder] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:13:48.489105 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-401f6762-f842-4359-9363-650e5f156d1f None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:13:48.493275 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71382b13-c0f7-4bb0-9662-ef3a7697efa8 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '847ec9a1bd4d4dec804318a5c845895a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417e3e2f1333481795a089bef2e24bea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:13:48.493974 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71382b13-c0f7-4bb0-9662-ef3a7697efa8 service neutron] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:13:48.494300 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71382b13-c0f7-4bb0-9662-ef3a7697efa8 service neutron] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:13:48.494615 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71382b13-c0f7-4bb0-9662-ef3a7697efa8 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:13:48.495945 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 981/1958] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jul 13 22:13:48 2022] GET /identity/v3/auth/tokens => generated 3266 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 13 22:13:48.504490 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71382b13-c0f7-4bb0-9662-ef3a7697efa8 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:13:48.505997 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71382b13-c0f7-4bb0-9662-ef3a7697efa8 service neutron] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:13:48.517662 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 978/1959] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jul 13 22:13:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 13 22:13:48.832774 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7713e05a-f2ac-458a-b6ea-fb8e30a688fa service cinder] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:13:48.833734 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-7713e05a-f2ac-458a-b6ea-fb8e30a688fa service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:13:48.858643 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7713e05a-f2ac-458a-b6ea-fb8e30a688fa service cinder] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:13:48.874112 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662a8b55-a5de-421a-9ff1-cdeda56abcac admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:13:48.875102 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662a8b55-a5de-421a-9ff1-cdeda56abcac admin admin] REQUEST_METHOD: `DELETE` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:13:48.875377 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662a8b55-a5de-421a-9ff1-cdeda56abcac admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:13:48.875647 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-662a8b55-a5de-421a-9ff1-cdeda56abcac admin admin] PATH_INFO: `/v3/projects/ccfa084ce0f14f92b7458b4e21df297c` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:13:48.876976 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-662a8b55-a5de-421a-9ff1-cdeda56abcac admin admin] RBAC: Authorizing `identity:delete_project(project_id=ccfa084ce0f14f92b7458b4e21df297c)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:13:48.879028 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-662a8b55-a5de-421a-9ff1-cdeda56abcac admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:13:48.931601 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-662a8b55-a5de-421a-9ff1-cdeda56abcac admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ccfa084ce0f14f92b7458b4e21df297c', 'request_id': 'req-662a8b55-a5de-421a-9ff1-cdeda56abcac'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:13:48.938419 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-662a8b55-a5de-421a-9ff1-cdeda56abcac admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ccfa084ce0f14f92b7458b4e21df297c', 'request_id': 'req-662a8b55-a5de-421a-9ff1-cdeda56abcac'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:13:48.944360 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-662a8b55-a5de-421a-9ff1-cdeda56abcac admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ccfa084ce0f14f92b7458b4e21df297c', 'request_id': 'req-662a8b55-a5de-421a-9ff1-cdeda56abcac'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:13:48.962726 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-662a8b55-a5de-421a-9ff1-cdeda56abcac admin admin] The token cache is being invalidate because project ccfa084ce0f14f92b7458b4e21df297c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80381) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 22:13:48.962726 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-662a8b55-a5de-421a-9ff1-cdeda56abcac admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:13:48.962726 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-662a8b55-a5de-421a-9ff1-cdeda56abcac admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:13:48.962726 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 982/1960] 10.222.0.52 () {58 vars in 1232 bytes} [Wed Jul 13 22:13:48 2022] DELETE /identity/v3/projects/ccfa084ce0f14f92b7458b4e21df297c => generated 0 bytes in 130 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 22:13:48.966230 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71382b13-c0f7-4bb0-9662-ef3a7697efa8 service neutron] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:13:48.989722 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71382b13-c0f7-4bb0-9662-ef3a7697efa8 service neutron] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:13:49.062418 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-caa13d53-4a63-4d2a-99bd-2e8ac0d3bbfd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:13:49.063423 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa13d53-4a63-4d2a-99bd-2e8ac0d3bbfd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:13:49.063704 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa13d53-4a63-4d2a-99bd-2e8ac0d3bbfd admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:13:49.063976 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-caa13d53-4a63-4d2a-99bd-2e8ac0d3bbfd admin admin] PATH_INFO: `/v3/users/8cdb18111a6d462ab779ad057c11c122` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:13:49.065205 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa13d53-4a63-4d2a-99bd-2e8ac0d3bbfd admin admin] RBAC: Authorizing `identity:delete_user(user_id=8cdb18111a6d462ab779ad057c11c122)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:13:49.067334 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-caa13d53-4a63-4d2a-99bd-2e8ac0d3bbfd admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:13:49.119151 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-caa13d53-4a63-4d2a-99bd-2e8ac0d3bbfd admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8cdb18111a6d462ab779ad057c11c122'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:13:49.129217 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-caa13d53-4a63-4d2a-99bd-2e8ac0d3bbfd admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8cdb18111a6d462ab779ad057c11c122'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:13:49.140961 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-caa13d53-4a63-4d2a-99bd-2e8ac0d3bbfd admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8cdb18111a6d462ab779ad057c11c122'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:13:49.141585 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-caa13d53-4a63-4d2a-99bd-2e8ac0d3bbfd admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8cdb18111a6d462ab779ad057c11c122'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:13:49.142174 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-caa13d53-4a63-4d2a-99bd-2e8ac0d3bbfd admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8cdb18111a6d462ab779ad057c11c122'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:13:49.154025 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 979/1961] 10.222.0.52 () {58 vars in 1223 bytes} [Wed Jul 13 22:13:48 2022] DELETE /identity/v3/users/8cdb18111a6d462ab779ad057c11c122 => generated 0 bytes in 189 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 22:13:49.211607 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662a8b55-a5de-421a-9ff1-cdeda56abcac admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:13:49.213322 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-662a8b55-a5de-421a-9ff1-cdeda56abcac admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:13:49.227145 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-662a8b55-a5de-421a-9ff1-cdeda56abcac admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:13:49.248107 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:13:49.289068 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9cb48c-62d9-4486-bf7b-68c00adeaeba admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:13:49.289967 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9cb48c-62d9-4486-bf7b-68c00adeaeba admin admin] REQUEST_METHOD: `DELETE` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:13:49.290256 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9cb48c-62d9-4486-bf7b-68c00adeaeba admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:13:49.290564 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8b9cb48c-62d9-4486-bf7b-68c00adeaeba admin admin] PATH_INFO: `/v3/users/b710832fe67b4d7fb2ca9972372a282a` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:13:49.291752 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9cb48c-62d9-4486-bf7b-68c00adeaeba admin admin] RBAC: Authorizing `identity:delete_user(user_id=b710832fe67b4d7fb2ca9972372a282a)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:13:49.293730 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8b9cb48c-62d9-4486-bf7b-68c00adeaeba admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:13:49.347407 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-8b9cb48c-62d9-4486-bf7b-68c00adeaeba admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b710832fe67b4d7fb2ca9972372a282a'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:13:49.356381 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-8b9cb48c-62d9-4486-bf7b-68c00adeaeba admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b710832fe67b4d7fb2ca9972372a282a'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:13:49.369303 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-8b9cb48c-62d9-4486-bf7b-68c00adeaeba admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b710832fe67b4d7fb2ca9972372a282a'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:13:49.369912 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-8b9cb48c-62d9-4486-bf7b-68c00adeaeba admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b710832fe67b4d7fb2ca9972372a282a'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:13:49.370571 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-8b9cb48c-62d9-4486-bf7b-68c00adeaeba admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b710832fe67b4d7fb2ca9972372a282a'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:13:49.387086 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 983/1962] 10.222.0.52 () {58 vars in 1223 bytes} [Wed Jul 13 22:13:49 2022] DELETE /identity/v3/users/b710832fe67b4d7fb2ca9972372a282a => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 13 22:13:49.391343 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332172c7-c084-45a6-8337-d49210ef8177 None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:13:49.391817 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332172c7-c084-45a6-8337-d49210ef8177 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:13:49.391941 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-332172c7-c084-45a6-8337-d49210ef8177 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:13:49.409762 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-332172c7-c084-45a6-8337-d49210ef8177 None None] MFA Rules not processed for user `cc99c93da5074cf5a48bc2aa9a92c5e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 22:13:49.411002 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-332172c7-c084-45a6-8337-d49210ef8177 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:13:49.444481 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:13:49.467551 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9cb48c-62d9-4486-bf7b-68c00adeaeba admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:13:49.469576 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-8b9cb48c-62d9-4486-bf7b-68c00adeaeba admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:13:49.516740 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 980/1963] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jul 13 22:13:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 126 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 22:13:49.538554 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332172c7-c084-45a6-8337-d49210ef8177 None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:13:49.540596 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-332172c7-c084-45a6-8337-d49210ef8177 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:13:49.553651 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8b9cb48c-62d9-4486-bf7b-68c00adeaeba admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:13:49.560471 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3262e24c-3fa0-4a66-b0f4-620e6a911c76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:13:49.561677 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3262e24c-3fa0-4a66-b0f4-620e6a911c76 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:13:49.562014 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3262e24c-3fa0-4a66-b0f4-620e6a911c76 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:13:49.562298 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3262e24c-3fa0-4a66-b0f4-620e6a911c76 admin admin] PATH_INFO: `/v3/projects/211150b302aa41449f963175857b9c9f` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:13:49.563630 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3262e24c-3fa0-4a66-b0f4-620e6a911c76 admin admin] RBAC: Authorizing `identity:delete_project(project_id=211150b302aa41449f963175857b9c9f)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:13:49.566110 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3262e24c-3fa0-4a66-b0f4-620e6a911c76 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:13:49.609254 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-332172c7-c084-45a6-8337-d49210ef8177 None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:13:49.625636 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=86,keystone:DELETE=13,keystone:INSERT=1 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:13:49.630229 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-3262e24c-3fa0-4a66-b0f4-620e6a911c76 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '211150b302aa41449f963175857b9c9f', 'request_id': 'req-3262e24c-3fa0-4a66-b0f4-620e6a911c76'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:13:49.639454 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-3262e24c-3fa0-4a66-b0f4-620e6a911c76 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '211150b302aa41449f963175857b9c9f', 'request_id': 'req-3262e24c-3fa0-4a66-b0f4-620e6a911c76'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:13:49.646709 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-3262e24c-3fa0-4a66-b0f4-620e6a911c76 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '211150b302aa41449f963175857b9c9f', 'request_id': 'req-3262e24c-3fa0-4a66-b0f4-620e6a911c76'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:13:49.661309 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-3262e24c-3fa0-4a66-b0f4-620e6a911c76 admin admin] The token cache is being invalidate because project 211150b302aa41449f963175857b9c9f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80381) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 22:13:49.661653 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-3262e24c-3fa0-4a66-b0f4-620e6a911c76 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:13:49.662862 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-3262e24c-3fa0-4a66-b0f4-620e6a911c76 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:13:49.664546 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 984/1964] 10.222.0.52 () {58 vars in 1232 bytes} [Wed Jul 13 22:13:49 2022] DELETE /identity/v3/projects/211150b302aa41449f963175857b9c9f => generated 0 bytes in 198 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 13 22:13:49.669237 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032b0a17-3947-4504-aeb5-bb6b5af9b452 None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:13:49.669529 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032b0a17-3947-4504-aeb5-bb6b5af9b452 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:13:49.669807 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-032b0a17-3947-4504-aeb5-bb6b5af9b452 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:13:49.678732 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c14973-ccee-427d-81f4-fc673131c955 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '847ec9a1bd4d4dec804318a5c845895a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417e3e2f1333481795a089bef2e24bea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:13:49.679510 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c14973-ccee-427d-81f4-fc673131c955 service neutron] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:13:49.679794 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c14973-ccee-427d-81f4-fc673131c955 service neutron] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:13:49.680068 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2c14973-ccee-427d-81f4-fc673131c955 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:13:49.681832 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-a2c14973-ccee-427d-81f4-fc673131c955 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:13:49.687685 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-032b0a17-3947-4504-aeb5-bb6b5af9b452 None None] MFA Rules not processed for user `cc99c93da5074cf5a48bc2aa9a92c5e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 22:13:49.688866 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-032b0a17-3947-4504-aeb5-bb6b5af9b452 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:13:49.750829 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 985/1965] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jul 13 22:13:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 13 22:13:49.754788 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c14973-ccee-427d-81f4-fc673131c955 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:13:49.757075 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2c14973-ccee-427d-81f4-fc673131c955 service neutron] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:13:49.758601 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01722b85-4303-45b6-9766-e822569b5757 None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:13:49.758961 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01722b85-4303-45b6-9766-e822569b5757 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:13:49.759307 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-01722b85-4303-45b6-9766-e822569b5757 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:13:49.768450 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 981/1966] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jul 13 22:13:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 231 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 13 22:13:49.777807 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-01722b85-4303-45b6-9766-e822569b5757 None None] MFA Rules not processed for user `cc99c93da5074cf5a48bc2aa9a92c5e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 22:13:49.779058 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-01722b85-4303-45b6-9766-e822569b5757 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:13:49.786952 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 986/1967] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jul 13 22:13:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 13 22:13:49.791193 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c14973-ccee-427d-81f4-fc673131c955 service neutron] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:13:49.800771 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2c14973-ccee-427d-81f4-fc673131c955 service neutron] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:13:49.805337 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519ce73c-614b-4128-86a3-7f7ec00493d1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:13:49.806000 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519ce73c-614b-4128-86a3-7f7ec00493d1 admin admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:13:49.806286 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519ce73c-614b-4128-86a3-7f7ec00493d1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:13:49.806586 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-519ce73c-614b-4128-86a3-7f7ec00493d1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:13:49.807316 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519ce73c-614b-4128-86a3-7f7ec00493d1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 22:13:49.807615 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519ce73c-614b-4128-86a3-7f7ec00493d1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:13:49.809213 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-519ce73c-614b-4128-86a3-7f7ec00493d1 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:13:49.821267 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 982/1968] 10.222.0.52 () {58 vars in 1152 bytes} [Wed Jul 13 22:13:49 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 22:13:49.826036 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01722b85-4303-45b6-9766-e822569b5757 None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:13:49.836252 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-01722b85-4303-45b6-9766-e822569b5757 None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:13:49.841681 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957ad078-0e41-474b-ae5e-500d4bbe7d3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:13:49.842355 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957ad078-0e41-474b-ae5e-500d4bbe7d3b admin admin] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:13:49.842668 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957ad078-0e41-474b-ae5e-500d4bbe7d3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:13:49.842951 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-957ad078-0e41-474b-ae5e-500d4bbe7d3b admin admin] PATH_INFO: `/v3/projects` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:13:49.843639 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957ad078-0e41-474b-ae5e-500d4bbe7d3b admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:13:49.845560 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-957ad078-0e41-474b-ae5e-500d4bbe7d3b admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:13:49.875172 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 987/1969] 10.222.0.52 () {60 vars in 1152 bytes} [Wed Jul 13 22:13:49 2022] POST /identity/v3/projects => generated 379 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 13 22:13:49.879350 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519ce73c-614b-4128-86a3-7f7ec00493d1 admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:13:49.889520 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-519ce73c-614b-4128-86a3-7f7ec00493d1 admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:13:49.929438 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:13:49.986047 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fdd97e2-e5e3-441c-a08e-09c17cdc2ea4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:13:49.987151 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdd97e2-e5e3-441c-a08e-09c17cdc2ea4 admin admin] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:13:49.987569 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdd97e2-e5e3-441c-a08e-09c17cdc2ea4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:13:49.988540 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0fdd97e2-e5e3-441c-a08e-09c17cdc2ea4 admin admin] PATH_INFO: `/v3/users` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:13:49.989363 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fdd97e2-e5e3-441c-a08e-09c17cdc2ea4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:13:49.992229 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0fdd97e2-e5e3-441c-a08e-09c17cdc2ea4 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:13:50.017385 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 983/1970] 10.222.0.52 () {60 vars in 1143 bytes} [Wed Jul 13 22:13:49 2022] POST /identity/v3/users => generated 290 bytes in 139 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 22:13:50.021767 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957ad078-0e41-474b-ae5e-500d4bbe7d3b admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:13:50.035153 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-957ad078-0e41-474b-ae5e-500d4bbe7d3b admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:13:50.041463 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67a4ed5-5636-40d0-a03c-ceced8412b45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:13:50.042242 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67a4ed5-5636-40d0-a03c-ceced8412b45 admin admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:13:50.042785 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67a4ed5-5636-40d0-a03c-ceced8412b45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:13:50.043133 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b67a4ed5-5636-40d0-a03c-ceced8412b45 admin admin] PATH_INFO: `/v3/roles` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:13:50.043954 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67a4ed5-5636-40d0-a03c-ceced8412b45 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:13:50.045669 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b67a4ed5-5636-40d0-a03c-ceced8412b45 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:13:50.058451 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 988/1971] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jul 13 22:13:50 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 22:13:50.062566 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fdd97e2-e5e3-441c-a08e-09c17cdc2ea4 admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:13:50.071186 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67a4ed5-5636-40d0-a03c-ceced8412b45 admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:13:50.073739 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-b67a4ed5-5636-40d0-a03c-ceced8412b45 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:13:50.082624 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0fdd97e2-e5e3-441c-a08e-09c17cdc2ea4 admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:13:50.088793 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da46479-9565-4808-926f-2cf707c5011b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:13:50.089485 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da46479-9565-4808-926f-2cf707c5011b admin admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:13:50.089767 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da46479-9565-4808-926f-2cf707c5011b admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:13:50.090046 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8da46479-9565-4808-926f-2cf707c5011b admin admin] PATH_INFO: `/v3/roles` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:13:50.090824 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da46479-9565-4808-926f-2cf707c5011b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:13:50.092423 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8da46479-9565-4808-926f-2cf707c5011b admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:13:50.092912 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b67a4ed5-5636-40d0-a03c-ceced8412b45 admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:13:50.099085 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c8d72c7-b6ef-4aae-98e4-dc1b358feed6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:13:50.100116 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8d72c7-b6ef-4aae-98e4-dc1b358feed6 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:13:50.100398 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8d72c7-b6ef-4aae-98e4-dc1b358feed6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:13:50.100750 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c8d72c7-b6ef-4aae-98e4-dc1b358feed6 admin admin] PATH_INFO: `/v3/projects/519ca221e7e7421581283c9d55a8c464` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:13:50.102212 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c8d72c7-b6ef-4aae-98e4-dc1b358feed6 admin admin] RBAC: Authorizing `identity:delete_project(project_id=519ca221e7e7421581283c9d55a8c464)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:13:50.104190 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c8d72c7-b6ef-4aae-98e4-dc1b358feed6 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:13:50.125605 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 984/1972] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jul 13 22:13:50 2022] GET /identity/v3/roles => generated 1420 bytes in 61 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 22:13:50.126710 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da46479-9565-4808-926f-2cf707c5011b admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:13:50.139970 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8da46479-9565-4808-926f-2cf707c5011b admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:13:50.145863 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad51c52c-800d-4562-8931-02634c8ef3a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:13:50.146810 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad51c52c-800d-4562-8931-02634c8ef3a3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:13:50.147096 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad51c52c-800d-4562-8931-02634c8ef3a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:13:50.147378 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad51c52c-800d-4562-8931-02634c8ef3a3 admin admin] PATH_INFO: `/v3/projects/ea4bb4885ad0471c8d41785429f3cded/users/782118db40e84d38897c252d252d11c9/roles/2a5e86976394498abb6310697cdb3f8b` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:13:50.164252 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad51c52c-800d-4562-8931-02634c8ef3a3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ea4bb4885ad0471c8d41785429f3cded, user_id=782118db40e84d38897c252d252d11c9, role_id=2a5e86976394498abb6310697cdb3f8b)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:13:50.164489 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-8c8d72c7-b6ef-4aae-98e4-dc1b358feed6 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '519ca221e7e7421581283c9d55a8c464', 'request_id': 'req-8c8d72c7-b6ef-4aae-98e4-dc1b358feed6'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:13:50.166977 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ad51c52c-800d-4562-8931-02634c8ef3a3 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:13:50.172241 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-8c8d72c7-b6ef-4aae-98e4-dc1b358feed6 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '519ca221e7e7421581283c9d55a8c464', 'request_id': 'req-8c8d72c7-b6ef-4aae-98e4-dc1b358feed6'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:13:50.179789 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-8c8d72c7-b6ef-4aae-98e4-dc1b358feed6 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '519ca221e7e7421581283c9d55a8c464', 'request_id': 'req-8c8d72c7-b6ef-4aae-98e4-dc1b358feed6'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:13:50.183899 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 985/1973] 10.222.0.52 () {60 vars in 1482 bytes} [Wed Jul 13 22:13:50 2022] PUT /identity/v3/projects/ea4bb4885ad0471c8d41785429f3cded/users/782118db40e84d38897c252d252d11c9/roles/2a5e86976394498abb6310697cdb3f8b => generated 0 bytes in 58 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 13 22:13:50.195261 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e765efa7-75cc-4000-ad6d-d892e88d6a35 None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:13:50.195567 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e765efa7-75cc-4000-ad6d-d892e88d6a35 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:13:50.195818 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e765efa7-75cc-4000-ad6d-d892e88d6a35 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:13:50.198476 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-8c8d72c7-b6ef-4aae-98e4-dc1b358feed6 admin admin] The token cache is being invalidate because project 519ca221e7e7421581283c9d55a8c464 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80381) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 22:13:50.198868 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-8c8d72c7-b6ef-4aae-98e4-dc1b358feed6 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:13:50.199616 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-8c8d72c7-b6ef-4aae-98e4-dc1b358feed6 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:13:50.201562 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 989/1974] 10.222.0.52 () {58 vars in 1232 bytes} [Wed Jul 13 22:13:50 2022] DELETE /identity/v3/projects/519ca221e7e7421581283c9d55a8c464 => generated 0 bytes in 132 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 22:13:50.207373 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116cd124-a769-4e0c-9e0c-7c1ee558a5e7 None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:13:50.207816 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116cd124-a769-4e0c-9e0c-7c1ee558a5e7 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:13:50.208095 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-116cd124-a769-4e0c-9e0c-7c1ee558a5e7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:13:50.228470 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-e765efa7-75cc-4000-ad6d-d892e88d6a35 None None] MFA Rules not processed for user `782118db40e84d38897c252d252d11c9`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 22:13:50.229755 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-e765efa7-75cc-4000-ad6d-d892e88d6a35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:13:50.230932 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-116cd124-a769-4e0c-9e0c-7c1ee558a5e7 None None] MFA Rules not processed for user `cc99c93da5074cf5a48bc2aa9a92c5e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 22:13:50.232096 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-116cd124-a769-4e0c-9e0c-7c1ee558a5e7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:13:50.245381 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:DELETE=11,keystone:SELECT=86,keystone:INSERT=3 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:13:50.311958 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 990/1975] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jul 13 22:13:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 22:13:50.317666 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 986/1976] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jul 13 22:13:50 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 123 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 22:13:50.320358 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3560f761-fa28-4c71-b774-53a989c4ec97 None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:13:50.320642 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3560f761-fa28-4c71-b774-53a989c4ec97 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:13:50.321002 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3560f761-fa28-4c71-b774-53a989c4ec97 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:13:50.322590 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e765efa7-75cc-4000-ad6d-d892e88d6a35 None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:13:50.324169 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-e765efa7-75cc-4000-ad6d-d892e88d6a35 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:13:50.340233 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e765efa7-75cc-4000-ad6d-d892e88d6a35 None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:13:50.340813 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-3560f761-fa28-4c71-b774-53a989c4ec97 None None] MFA Rules not processed for user `cc99c93da5074cf5a48bc2aa9a92c5e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 22:13:50.342011 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-3560f761-fa28-4c71-b774-53a989c4ec97 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:13:50.345166 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837f3d55-582c-4f98-91de-08525a4223f4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:13:50.345834 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837f3d55-582c-4f98-91de-08525a4223f4 admin admin] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:13:50.346170 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837f3d55-582c-4f98-91de-08525a4223f4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:13:50.346708 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-837f3d55-582c-4f98-91de-08525a4223f4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:13:50.347411 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837f3d55-582c-4f98-91de-08525a4223f4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:13:50.349340 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-837f3d55-582c-4f98-91de-08525a4223f4 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:13:50.350070 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 991/1977] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jul 13 22:13:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 22:13:50.354114 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3560f761-fa28-4c71-b774-53a989c4ec97 None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:13:50.363369 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3560f761-fa28-4c71-b774-53a989c4ec97 None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:13:50.368471 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a0e3cc-5461-4b56-b4cc-9aae07233fb9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:13:50.369124 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a0e3cc-5461-4b56-b4cc-9aae07233fb9 admin admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:13:50.369404 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a0e3cc-5461-4b56-b4cc-9aae07233fb9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:13:50.369675 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9a0e3cc-5461-4b56-b4cc-9aae07233fb9 admin admin] PATH_INFO: `/v3/domains` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:13:50.370401 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a0e3cc-5461-4b56-b4cc-9aae07233fb9 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 22:13:50.370735 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a0e3cc-5461-4b56-b4cc-9aae07233fb9 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:13:50.372499 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9a0e3cc-5461-4b56-b4cc-9aae07233fb9 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:13:50.376071 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 987/1978] 10.222.0.52 () {60 vars in 1152 bytes} [Wed Jul 13 22:13:50 2022] POST /identity/v3/projects => generated 379 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 13 22:13:50.380138 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837f3d55-582c-4f98-91de-08525a4223f4 admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:13:50.385374 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 992/1979] 10.222.0.52 () {58 vars in 1152 bytes} [Wed Jul 13 22:13:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 22:13:50.389711 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a0e3cc-5461-4b56-b4cc-9aae07233fb9 admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:13:50.399973 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9a0e3cc-5461-4b56-b4cc-9aae07233fb9 admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:13:50.424377 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:13:50.442076 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-837f3d55-582c-4f98-91de-08525a4223f4 admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:13:50.448063 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac9670a-14de-4c57-abcd-30b28d0d1ddf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:13:50.448743 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac9670a-14de-4c57-abcd-30b28d0d1ddf admin admin] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:13:50.449051 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac9670a-14de-4c57-abcd-30b28d0d1ddf admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:13:50.449321 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ac9670a-14de-4c57-abcd-30b28d0d1ddf admin admin] PATH_INFO: `/v3/users` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:13:50.450022 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac9670a-14de-4c57-abcd-30b28d0d1ddf admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:13:50.452002 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1ac9670a-14de-4c57-abcd-30b28d0d1ddf admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:13:50.456006 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d2e8ab-fad3-4b51-9ec5-e29a9d5ebd16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:13:50.456673 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d2e8ab-fad3-4b51-9ec5-e29a9d5ebd16 admin admin] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:13:50.456950 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d2e8ab-fad3-4b51-9ec5-e29a9d5ebd16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:13:50.457221 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59d2e8ab-fad3-4b51-9ec5-e29a9d5ebd16 admin admin] PATH_INFO: `/v3/projects` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:13:50.457905 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d2e8ab-fad3-4b51-9ec5-e29a9d5ebd16 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:13:50.460413 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-59d2e8ab-fad3-4b51-9ec5-e29a9d5ebd16 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:13:50.474048 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 988/1980] 10.222.0.52 () {60 vars in 1143 bytes} [Wed Jul 13 22:13:50 2022] POST /identity/v3/users => generated 296 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 22:13:50.477964 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac9670a-14de-4c57-abcd-30b28d0d1ddf admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:13:50.487850 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ac9670a-14de-4c57-abcd-30b28d0d1ddf admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:13:50.492084 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 993/1981] 10.222.0.52 () {60 vars in 1152 bytes} [Wed Jul 13 22:13:50 2022] POST /identity/v3/projects => generated 373 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 13 22:13:50.492997 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d2e8ab-fad3-4b51-9ec5-e29a9d5ebd16 admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:13:50.549141 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12de0510-9885-4f1c-b03e-48b7415c00a7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:13:50.549884 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12de0510-9885-4f1c-b03e-48b7415c00a7 admin admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:13:50.550160 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12de0510-9885-4f1c-b03e-48b7415c00a7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:13:50.550432 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12de0510-9885-4f1c-b03e-48b7415c00a7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:13:50.551191 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12de0510-9885-4f1c-b03e-48b7415c00a7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:13:50.552801 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12de0510-9885-4f1c-b03e-48b7415c00a7 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:13:50.563301 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 989/1982] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jul 13 22:13:50 2022] GET /identity/v3/roles => generated 1420 bytes in 86 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 13 22:13:50.564271 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59d2e8ab-fad3-4b51-9ec5-e29a9d5ebd16 admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:13:50.567509 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12de0510-9885-4f1c-b03e-48b7415c00a7 admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:13:50.569674 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43736b25-aa78-4644-aaec-406c759ea7b2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:13:50.570359 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43736b25-aa78-4644-aaec-406c759ea7b2 admin admin] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:13:50.570678 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43736b25-aa78-4644-aaec-406c759ea7b2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:13:50.570957 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-43736b25-aa78-4644-aaec-406c759ea7b2 admin admin] PATH_INFO: `/v3/users` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:13:50.571650 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43736b25-aa78-4644-aaec-406c759ea7b2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:13:50.573921 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-43736b25-aa78-4644-aaec-406c759ea7b2 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:13:50.577491 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12de0510-9885-4f1c-b03e-48b7415c00a7 admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:13:50.582242 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2039473-90f3-4535-8f4c-430cbb80d896 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:13:50.583136 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2039473-90f3-4535-8f4c-430cbb80d896 admin admin] REQUEST_METHOD: `PUT` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:13:50.583417 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2039473-90f3-4535-8f4c-430cbb80d896 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:13:50.583686 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2039473-90f3-4535-8f4c-430cbb80d896 admin admin] PATH_INFO: `/v3/projects/8514586aa53141629da48ce21495e6e4/users/818166c1132d4011bfeb907555d20456/roles/44efba2579f342df8a0789c59d163ec3` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:13:50.599983 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 994/1983] 10.222.0.52 () {60 vars in 1143 bytes} [Wed Jul 13 22:13:50 2022] POST /identity/v3/users => generated 287 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 13 22:13:50.600170 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2039473-90f3-4535-8f4c-430cbb80d896 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8514586aa53141629da48ce21495e6e4, user_id=818166c1132d4011bfeb907555d20456, role_id=44efba2579f342df8a0789c59d163ec3)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:13:50.601089 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43736b25-aa78-4644-aaec-406c759ea7b2 admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:13:50.602191 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2039473-90f3-4535-8f4c-430cbb80d896 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:13:50.612206 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-43736b25-aa78-4644-aaec-406c759ea7b2 admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:13:50.618870 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 990/1984] 10.222.0.52 () {60 vars in 1482 bytes} [Wed Jul 13 22:13:50 2022] PUT /identity/v3/projects/8514586aa53141629da48ce21495e6e4/users/818166c1132d4011bfeb907555d20456/roles/44efba2579f342df8a0789c59d163ec3 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 13 22:13:50.622741 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2039473-90f3-4535-8f4c-430cbb80d896 admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:13:50.622917 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:13:50.682349 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207b94e8-97f0-45d0-b30b-189b56cad081 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:13:50.683307 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207b94e8-97f0-45d0-b30b-189b56cad081 admin admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:13:50.683881 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207b94e8-97f0-45d0-b30b-189b56cad081 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:13:50.684331 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-207b94e8-97f0-45d0-b30b-189b56cad081 admin admin] PATH_INFO: `/v3/roles` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:13:50.685224 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207b94e8-97f0-45d0-b30b-189b56cad081 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:13:50.686944 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-207b94e8-97f0-45d0-b30b-189b56cad081 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:13:50.699346 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 995/1985] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jul 13 22:13:50 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 13 22:13:50.703449 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207b94e8-97f0-45d0-b30b-189b56cad081 admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:13:50.705158 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2039473-90f3-4535-8f4c-430cbb80d896 admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:13:50.710387 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef0c33f-36af-40db-8058-f91c58e851f0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:13:50.711107 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef0c33f-36af-40db-8058-f91c58e851f0 admin admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:13:50.711518 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef0c33f-36af-40db-8058-f91c58e851f0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:13:50.711805 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ef0c33f-36af-40db-8058-f91c58e851f0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:13:50.712550 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef0c33f-36af-40db-8058-f91c58e851f0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:13:50.714205 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ef0c33f-36af-40db-8058-f91c58e851f0 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:13:50.715310 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-207b94e8-97f0-45d0-b30b-189b56cad081 admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:13:50.720870 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40ab6b3-1f71-4ac6-a028-fc7c1b2cab16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:13:50.721663 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40ab6b3-1f71-4ac6-a028-fc7c1b2cab16 admin admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:13:50.721950 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40ab6b3-1f71-4ac6-a028-fc7c1b2cab16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:13:50.722224 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f40ab6b3-1f71-4ac6-a028-fc7c1b2cab16 admin admin] PATH_INFO: `/v3/roles` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:13:50.723116 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f40ab6b3-1f71-4ac6-a028-fc7c1b2cab16 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:13:50.724719 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f40ab6b3-1f71-4ac6-a028-fc7c1b2cab16 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:13:50.726686 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 991/1986] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jul 13 22:13:50 2022] GET /identity/v3/roles => generated 1420 bytes in 104 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 13 22:13:50.730519 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef0c33f-36af-40db-8058-f91c58e851f0 admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:13:50.735540 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 996/1987] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jul 13 22:13:50 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 13 22:13:50.739821 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40ab6b3-1f71-4ac6-a028-fc7c1b2cab16 admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:13:50.741584 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ef0c33f-36af-40db-8058-f91c58e851f0 admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:13:50.746609 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a4536c9-7a3c-4d8b-9243-ffdce3e7fc81 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:13:50.747631 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4536c9-7a3c-4d8b-9243-ffdce3e7fc81 admin admin] REQUEST_METHOD: `PUT` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:13:50.747914 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4536c9-7a3c-4d8b-9243-ffdce3e7fc81 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:13:50.748315 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a4536c9-7a3c-4d8b-9243-ffdce3e7fc81 admin admin] PATH_INFO: `/v3/projects/8514586aa53141629da48ce21495e6e4/users/818166c1132d4011bfeb907555d20456/roles/44efba2579f342df8a0789c59d163ec3` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:13:50.750520 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4536c9-7a3c-4d8b-9243-ffdce3e7fc81 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8514586aa53141629da48ce21495e6e4, user_id=818166c1132d4011bfeb907555d20456, role_id=44efba2579f342df8a0789c59d163ec3)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:13:50.753553 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a4536c9-7a3c-4d8b-9243-ffdce3e7fc81 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:13:50.753714 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f40ab6b3-1f71-4ac6-a028-fc7c1b2cab16 admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:13:50.760000 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e18ea759-3c2f-4a61-87a8-63478e2e4e2f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:13:50.760913 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18ea759-3c2f-4a61-87a8-63478e2e4e2f admin admin] REQUEST_METHOD: `PUT` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:13:50.761201 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18ea759-3c2f-4a61-87a8-63478e2e4e2f admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:13:50.761476 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e18ea759-3c2f-4a61-87a8-63478e2e4e2f admin admin] PATH_INFO: `/v3/projects/9720d917aaf44278818af01a0c7cf546/users/9dfdd18b1fe342e78ffa119703695b11/roles/2a5e86976394498abb6310697cdb3f8b` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:13:50.768475 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 992/1988] 10.222.0.52 () {60 vars in 1482 bytes} [Wed Jul 13 22:13:50 2022] PUT /identity/v3/projects/8514586aa53141629da48ce21495e6e4/users/818166c1132d4011bfeb907555d20456/roles/44efba2579f342df8a0789c59d163ec3 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 13 22:13:50.781262 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6e77e6-7cbe-4250-974c-e949e85e3bb7 None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:13:50.781885 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6e77e6-7cbe-4250-974c-e949e85e3bb7 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:13:50.782780 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b6e77e6-7cbe-4250-974c-e949e85e3bb7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:13:50.783778 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18ea759-3c2f-4a61-87a8-63478e2e4e2f admin admin] RBAC: Authorizing `identity:create_grant(project_id=9720d917aaf44278818af01a0c7cf546, user_id=9dfdd18b1fe342e78ffa119703695b11, role_id=2a5e86976394498abb6310697cdb3f8b)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:13:50.786678 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e18ea759-3c2f-4a61-87a8-63478e2e4e2f admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:13:50.810729 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 997/1989] 10.222.0.52 () {60 vars in 1482 bytes} [Wed Jul 13 22:13:50 2022] PUT /identity/v3/projects/9720d917aaf44278818af01a0c7cf546/users/9dfdd18b1fe342e78ffa119703695b11/roles/2a5e86976394498abb6310697cdb3f8b => generated 0 bytes in 71 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 13 22:13:50.823038 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fb07f7-9b2d-45d7-9a87-5af240b5e4be None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:13:50.823326 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fb07f7-9b2d-45d7-9a87-5af240b5e4be None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:13:50.823424 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-5b6e77e6-7cbe-4250-974c-e949e85e3bb7 None None] MFA Rules not processed for user `818166c1132d4011bfeb907555d20456`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 22:13:50.823600 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7fb07f7-9b2d-45d7-9a87-5af240b5e4be None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:13:50.824690 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-5b6e77e6-7cbe-4250-974c-e949e85e3bb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:13:50.841707 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:13:50.850597 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-b7fb07f7-9b2d-45d7-9a87-5af240b5e4be None None] MFA Rules not processed for user `9dfdd18b1fe342e78ffa119703695b11`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 22:13:50.851818 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-b7fb07f7-9b2d-45d7-9a87-5af240b5e4be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:13:50.895514 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 993/1990] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jul 13 22:13:50 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 22:13:50.907661 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6e77e6-7cbe-4250-974c-e949e85e3bb7 None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:13:50.909158 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-5b6e77e6-7cbe-4250-974c-e949e85e3bb7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:13:50.918804 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 998/1991] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jul 13 22:13:50 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 13 22:13:50.930353 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7fb07f7-9b2d-45d7-9a87-5af240b5e4be None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:13:50.931793 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-b7fb07f7-9b2d-45d7-9a87-5af240b5e4be None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:13:50.972904 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b6e77e6-7cbe-4250-974c-e949e85e3bb7 None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:13:50.976937 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d53e44-9792-4780-b703-ee999181dc3a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc77e1ea1ada4700b809beb1c980c5e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417e3e2f1333481795a089bef2e24bea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:13:50.977643 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d53e44-9792-4780-b703-ee999181dc3a service cinder] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:13:50.977926 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d53e44-9792-4780-b703-ee999181dc3a service cinder] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:13:50.978188 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2d53e44-9792-4780-b703-ee999181dc3a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:13:50.988430 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d53e44-9792-4780-b703-ee999181dc3a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:13:50.989890 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2d53e44-9792-4780-b703-ee999181dc3a service cinder] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:13:50.996632 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7fb07f7-9b2d-45d7-9a87-5af240b5e4be None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:13:50.999666 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 994/1992] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jul 13 22:13:50 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 13 22:13:51.000200 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f3189c82-55d2-4a2b-909a-1166fa8cc7c5 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc77e1ea1ada4700b809beb1c980c5e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417e3e2f1333481795a089bef2e24bea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:13:51.000913 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3189c82-55d2-4a2b-909a-1166fa8cc7c5 service cinder] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:13:51.001202 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3189c82-55d2-4a2b-909a-1166fa8cc7c5 service cinder] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:13:51.001476 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3189c82-55d2-4a2b-909a-1166fa8cc7c5 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:13:51.010615 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3189c82-55d2-4a2b-909a-1166fa8cc7c5 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:13:51.012064 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f3189c82-55d2-4a2b-909a-1166fa8cc7c5 service cinder] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:13:51.021541 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 999/1993] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jul 13 22:13:50 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 13 22:13:51.051197 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d53e44-9792-4780-b703-ee999181dc3a service cinder] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:13:51.134573 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2d53e44-9792-4780-b703-ee999181dc3a service cinder] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:13:51.139581 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb74ae6-fc7a-43e3-8ee1-9064115e4a5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:13:51.140546 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb74ae6-fc7a-43e3-8ee1-9064115e4a5c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:13:51.140914 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb74ae6-fc7a-43e3-8ee1-9064115e4a5c admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:13:51.141252 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cb74ae6-fc7a-43e3-8ee1-9064115e4a5c admin admin] PATH_INFO: `/v3/users/9dfdd18b1fe342e78ffa119703695b11` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:13:51.142555 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb74ae6-fc7a-43e3-8ee1-9064115e4a5c admin admin] RBAC: Authorizing `identity:delete_user(user_id=9dfdd18b1fe342e78ffa119703695b11)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:13:51.144951 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cb74ae6-fc7a-43e3-8ee1-9064115e4a5c admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:13:51.201874 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-6cb74ae6-fc7a-43e3-8ee1-9064115e4a5c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '9dfdd18b1fe342e78ffa119703695b11'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:13:51.212879 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-6cb74ae6-fc7a-43e3-8ee1-9064115e4a5c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '9dfdd18b1fe342e78ffa119703695b11'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:13:51.218109 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:13:51.227854 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-6cb74ae6-fc7a-43e3-8ee1-9064115e4a5c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '9dfdd18b1fe342e78ffa119703695b11'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:13:51.228652 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-6cb74ae6-fc7a-43e3-8ee1-9064115e4a5c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '9dfdd18b1fe342e78ffa119703695b11'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:13:51.229385 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-6cb74ae6-fc7a-43e3-8ee1-9064115e4a5c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '9dfdd18b1fe342e78ffa119703695b11'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:13:51.246544 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 995/1994] 10.222.0.52 () {58 vars in 1223 bytes} [Wed Jul 13 22:13:51 2022] DELETE /identity/v3/users/9dfdd18b1fe342e78ffa119703695b11 => generated 0 bytes in 194 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 13 22:13:51.249137 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3293f7b7-7c89-45c8-88c7-e9ba78c2d985 None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:13:51.249641 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3293f7b7-7c89-45c8-88c7-e9ba78c2d985 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:13:51.249785 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3293f7b7-7c89-45c8-88c7-e9ba78c2d985 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:13:51.264031 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:13:51.271447 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-3293f7b7-7c89-45c8-88c7-e9ba78c2d985 None None] MFA Rules not processed for user `cc99c93da5074cf5a48bc2aa9a92c5e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 22:13:51.273045 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-3293f7b7-7c89-45c8-88c7-e9ba78c2d985 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:13:51.341701 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1000/1995] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jul 13 22:13:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 22:13:51.351598 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb74ae6-fc7a-43e3-8ee1-9064115e4a5c admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:13:51.353032 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-6cb74ae6-fc7a-43e3-8ee1-9064115e4a5c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:13:51.362103 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3293f7b7-7c89-45c8-88c7-e9ba78c2d985 None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:13:51.363490 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-3293f7b7-7c89-45c8-88c7-e9ba78c2d985 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:13:51.414771 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cb74ae6-fc7a-43e3-8ee1-9064115e4a5c admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:13:51.418586 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200947b4-199e-4c19-b81c-ee0991145e82 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '847ec9a1bd4d4dec804318a5c845895a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417e3e2f1333481795a089bef2e24bea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:13:51.419244 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200947b4-199e-4c19-b81c-ee0991145e82 service neutron] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:13:51.419518 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200947b4-199e-4c19-b81c-ee0991145e82 service neutron] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:13:51.419787 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-200947b4-199e-4c19-b81c-ee0991145e82 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:13:51.425960 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3293f7b7-7c89-45c8-88c7-e9ba78c2d985 None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:13:51.429887 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-200947b4-199e-4c19-b81c-ee0991145e82 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:13:51.430438 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751cbcc2-f1e0-40da-a314-2064474b22f6 tempest-VolumeQuotaClassesTest-792225830 tempest-VolumeQuotaClassesTest-792225830-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818166c1132d4011bfeb907555d20456', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8514586aa53141629da48ce21495e6e4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:13:51.431130 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751cbcc2-f1e0-40da-a314-2064474b22f6 tempest-VolumeQuotaClassesTest-792225830 tempest-VolumeQuotaClassesTest-792225830-project-admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:13:51.431395 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-200947b4-199e-4c19-b81c-ee0991145e82 service neutron] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:13:51.431489 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751cbcc2-f1e0-40da-a314-2064474b22f6 tempest-VolumeQuotaClassesTest-792225830 tempest-VolumeQuotaClassesTest-792225830-project-admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:13:51.431684 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-751cbcc2-f1e0-40da-a314-2064474b22f6 tempest-VolumeQuotaClassesTest-792225830 tempest-VolumeQuotaClassesTest-792225830-project-admin] PATH_INFO: `/v3/domains` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:13:51.432419 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751cbcc2-f1e0-40da-a314-2064474b22f6 tempest-VolumeQuotaClassesTest-792225830 tempest-VolumeQuotaClassesTest-792225830-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 22:13:51.432728 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751cbcc2-f1e0-40da-a314-2064474b22f6 tempest-VolumeQuotaClassesTest-792225830 tempest-VolumeQuotaClassesTest-792225830-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:13:51.434327 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-751cbcc2-f1e0-40da-a314-2064474b22f6 tempest-VolumeQuotaClassesTest-792225830 tempest-VolumeQuotaClassesTest-792225830-project-admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:13:51.441240 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 996/1996] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jul 13 22:13:51 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 22:13:51.446153 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1001/1997] 10.222.0.52 () {58 vars in 1152 bytes} [Wed Jul 13 22:13:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 85 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 22:13:51.450758 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200947b4-199e-4c19-b81c-ee0991145e82 service neutron] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:13:51.461711 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-200947b4-199e-4c19-b81c-ee0991145e82 service neutron] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:13:51.467132 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7951a72-da98-4214-8c4e-20773ec92058 tempest-VolumeQuotaClassesTest-792225830 tempest-VolumeQuotaClassesTest-792225830-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818166c1132d4011bfeb907555d20456', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8514586aa53141629da48ce21495e6e4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:13:51.467900 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7951a72-da98-4214-8c4e-20773ec92058 tempest-VolumeQuotaClassesTest-792225830 tempest-VolumeQuotaClassesTest-792225830-project-admin] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:13:51.468378 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7951a72-da98-4214-8c4e-20773ec92058 tempest-VolumeQuotaClassesTest-792225830 tempest-VolumeQuotaClassesTest-792225830-project-admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:13:51.468663 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7951a72-da98-4214-8c4e-20773ec92058 tempest-VolumeQuotaClassesTest-792225830 tempest-VolumeQuotaClassesTest-792225830-project-admin] PATH_INFO: `/v3/projects` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:13:51.469367 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7951a72-da98-4214-8c4e-20773ec92058 tempest-VolumeQuotaClassesTest-792225830 tempest-VolumeQuotaClassesTest-792225830-project-admin] RBAC: Authorizing `identity:create_project()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:13:51.471386 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7951a72-da98-4214-8c4e-20773ec92058 tempest-VolumeQuotaClassesTest-792225830 tempest-VolumeQuotaClassesTest-792225830-project-admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:13:51.500543 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 997/1998] 10.222.0.52 () {60 vars in 1152 bytes} [Wed Jul 13 22:13:51 2022] POST /identity/v3/projects => generated 353 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 13 22:13:51.504817 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751cbcc2-f1e0-40da-a314-2064474b22f6 tempest-VolumeQuotaClassesTest-792225830 tempest-VolumeQuotaClassesTest-792225830-project-admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:13:51.515933 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-751cbcc2-f1e0-40da-a314-2064474b22f6 tempest-VolumeQuotaClassesTest-792225830 tempest-VolumeQuotaClassesTest-792225830-project-admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:13:51.579726 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83b3b1e-e915-4f16-90f0-3bca587aea59 tempest-VolumeQuotaClassesTest-792225830 tempest-VolumeQuotaClassesTest-792225830-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818166c1132d4011bfeb907555d20456', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8514586aa53141629da48ce21495e6e4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:13:51.580348 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83b3b1e-e915-4f16-90f0-3bca587aea59 tempest-VolumeQuotaClassesTest-792225830 tempest-VolumeQuotaClassesTest-792225830-project-admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:13:51.580777 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83b3b1e-e915-4f16-90f0-3bca587aea59 tempest-VolumeQuotaClassesTest-792225830 tempest-VolumeQuotaClassesTest-792225830-project-admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:13:51.581066 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d83b3b1e-e915-4f16-90f0-3bca587aea59 tempest-VolumeQuotaClassesTest-792225830 tempest-VolumeQuotaClassesTest-792225830-project-admin] PATH_INFO: `/v3/domains` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:13:51.581886 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d83b3b1e-e915-4f16-90f0-3bca587aea59 tempest-VolumeQuotaClassesTest-792225830 tempest-VolumeQuotaClassesTest-792225830-project-admin] RBAC: Adding query filter params (name=Default) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 22:13:51.582196 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d83b3b1e-e915-4f16-90f0-3bca587aea59 tempest-VolumeQuotaClassesTest-792225830 tempest-VolumeQuotaClassesTest-792225830-project-admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:13:51.583936 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d83b3b1e-e915-4f16-90f0-3bca587aea59 tempest-VolumeQuotaClassesTest-792225830 tempest-VolumeQuotaClassesTest-792225830-project-admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:13:51.597139 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1002/1999] 10.222.0.52 () {58 vars in 1152 bytes} [Wed Jul 13 22:13:51 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 22:13:51.635505 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7951a72-da98-4214-8c4e-20773ec92058 tempest-VolumeQuotaClassesTest-792225830 tempest-VolumeQuotaClassesTest-792225830-project-admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:13:51.646600 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7951a72-da98-4214-8c4e-20773ec92058 tempest-VolumeQuotaClassesTest-792225830 tempest-VolumeQuotaClassesTest-792225830-project-admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:13:51.651227 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b7b68a7d-e74f-4111-9f71-0f4ab0a4fbf5 tempest-VolumeQuotaClassesTest-792225830 tempest-VolumeQuotaClassesTest-792225830-project-admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '818166c1132d4011bfeb907555d20456', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '8514586aa53141629da48ce21495e6e4', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:13:51.652118 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b68a7d-e74f-4111-9f71-0f4ab0a4fbf5 tempest-VolumeQuotaClassesTest-792225830 tempest-VolumeQuotaClassesTest-792225830-project-admin] REQUEST_METHOD: `DELETE` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:13:51.652400 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b68a7d-e74f-4111-9f71-0f4ab0a4fbf5 tempest-VolumeQuotaClassesTest-792225830 tempest-VolumeQuotaClassesTest-792225830-project-admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:13:51.652683 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b7b68a7d-e74f-4111-9f71-0f4ab0a4fbf5 tempest-VolumeQuotaClassesTest-792225830 tempest-VolumeQuotaClassesTest-792225830-project-admin] PATH_INFO: `/v3/projects/b02e2d772941418fa31a203c3bc9c0f2` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:13:51.653997 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b68a7d-e74f-4111-9f71-0f4ab0a4fbf5 tempest-VolumeQuotaClassesTest-792225830 tempest-VolumeQuotaClassesTest-792225830-project-admin] RBAC: Authorizing `identity:delete_project(project_id=b02e2d772941418fa31a203c3bc9c0f2)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:13:51.656036 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b7b68a7d-e74f-4111-9f71-0f4ab0a4fbf5 tempest-VolumeQuotaClassesTest-792225830 tempest-VolumeQuotaClassesTest-792225830-project-admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:13:51.709698 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-b7b68a7d-e74f-4111-9f71-0f4ab0a4fbf5 tempest-VolumeQuotaClassesTest-792225830 tempest-VolumeQuotaClassesTest-792225830-project-admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b02e2d772941418fa31a203c3bc9c0f2', 'request_id': 'req-b7b68a7d-e74f-4111-9f71-0f4ab0a4fbf5'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:13:51.716807 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-b7b68a7d-e74f-4111-9f71-0f4ab0a4fbf5 tempest-VolumeQuotaClassesTest-792225830 tempest-VolumeQuotaClassesTest-792225830-project-admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b02e2d772941418fa31a203c3bc9c0f2', 'request_id': 'req-b7b68a7d-e74f-4111-9f71-0f4ab0a4fbf5'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:13:51.722760 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-b7b68a7d-e74f-4111-9f71-0f4ab0a4fbf5 tempest-VolumeQuotaClassesTest-792225830 tempest-VolumeQuotaClassesTest-792225830-project-admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b02e2d772941418fa31a203c3bc9c0f2', 'request_id': 'req-b7b68a7d-e74f-4111-9f71-0f4ab0a4fbf5'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:13:51.728593 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83b3b1e-e915-4f16-90f0-3bca587aea59 tempest-VolumeQuotaClassesTest-792225830 tempest-VolumeQuotaClassesTest-792225830-project-admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:13:51.731137 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-d83b3b1e-e915-4f16-90f0-3bca587aea59 tempest-VolumeQuotaClassesTest-792225830 tempest-VolumeQuotaClassesTest-792225830-project-admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:13:51.739136 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-b7b68a7d-e74f-4111-9f71-0f4ab0a4fbf5 tempest-VolumeQuotaClassesTest-792225830 tempest-VolumeQuotaClassesTest-792225830-project-admin] The token cache is being invalidate because project b02e2d772941418fa31a203c3bc9c0f2 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80382) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 22:13:51.739523 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-b7b68a7d-e74f-4111-9f71-0f4ab0a4fbf5 tempest-VolumeQuotaClassesTest-792225830 tempest-VolumeQuotaClassesTest-792225830-project-admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:13:51.740351 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-b7b68a7d-e74f-4111-9f71-0f4ab0a4fbf5 tempest-VolumeQuotaClassesTest-792225830 tempest-VolumeQuotaClassesTest-792225830-project-admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:13:51.742232 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 998/2000] 10.222.0.52 () {58 vars in 1232 bytes} [Wed Jul 13 22:13:51 2022] DELETE /identity/v3/projects/b02e2d772941418fa31a203c3bc9c0f2 => generated 0 bytes in 107 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 22:13:51.782958 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:13:51.810671 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d83b3b1e-e915-4f16-90f0-3bca587aea59 tempest-VolumeQuotaClassesTest-792225830 tempest-VolumeQuotaClassesTest-792225830-project-admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:13:51.878885 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82850d31-60be-40ea-ba69-39bc46244c0f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:13:51.879951 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82850d31-60be-40ea-ba69-39bc46244c0f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:13:51.880376 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82850d31-60be-40ea-ba69-39bc46244c0f admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:13:51.880665 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82850d31-60be-40ea-ba69-39bc46244c0f admin admin] PATH_INFO: `/v3/projects/9720d917aaf44278818af01a0c7cf546` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:13:51.882568 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82850d31-60be-40ea-ba69-39bc46244c0f admin admin] RBAC: Authorizing `identity:delete_project(project_id=9720d917aaf44278818af01a0c7cf546)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:13:51.889354 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-82850d31-60be-40ea-ba69-39bc46244c0f admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:13:51.939285 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-82850d31-60be-40ea-ba69-39bc46244c0f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '9720d917aaf44278818af01a0c7cf546', 'request_id': 'req-82850d31-60be-40ea-ba69-39bc46244c0f'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:13:51.946347 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-82850d31-60be-40ea-ba69-39bc46244c0f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '9720d917aaf44278818af01a0c7cf546', 'request_id': 'req-82850d31-60be-40ea-ba69-39bc46244c0f'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:13:51.954069 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-82850d31-60be-40ea-ba69-39bc46244c0f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '9720d917aaf44278818af01a0c7cf546', 'request_id': 'req-82850d31-60be-40ea-ba69-39bc46244c0f'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:13:51.968987 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-82850d31-60be-40ea-ba69-39bc46244c0f admin admin] The token cache is being invalidate because project 9720d917aaf44278818af01a0c7cf546 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80381) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 22:13:51.969276 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-82850d31-60be-40ea-ba69-39bc46244c0f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:13:51.970249 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-82850d31-60be-40ea-ba69-39bc46244c0f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:13:51.971879 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1003/2001] 10.222.0.52 () {58 vars in 1232 bytes} [Wed Jul 13 22:13:51 2022] DELETE /identity/v3/projects/9720d917aaf44278818af01a0c7cf546 => generated 0 bytes in 244 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 22:13:51.976306 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82850d31-60be-40ea-ba69-39bc46244c0f admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:13:51.976852 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1951bcc8-5db8-4353-a8e9-791ec6640a7f None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:13:51.977343 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1951bcc8-5db8-4353-a8e9-791ec6640a7f None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:13:51.977846 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-82850d31-60be-40ea-ba69-39bc46244c0f admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:13:51.978086 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1951bcc8-5db8-4353-a8e9-791ec6640a7f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:13:52.003880 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-1951bcc8-5db8-4353-a8e9-791ec6640a7f None None] MFA Rules not processed for user `cc99c93da5074cf5a48bc2aa9a92c5e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 22:13:52.005506 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-1951bcc8-5db8-4353-a8e9-791ec6640a7f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:13:52.040689 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:DELETE=9,keystone:SELECT=89,keystone:INSERT=2 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:13:52.046032 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82850d31-60be-40ea-ba69-39bc46244c0f admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:13:52.051200 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4137b9-03d1-422c-af07-361998d2c296 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:13:52.052085 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4137b9-03d1-422c-af07-361998d2c296 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:13:52.052360 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4137b9-03d1-422c-af07-361998d2c296 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:13:52.052633 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cf4137b9-03d1-422c-af07-361998d2c296 admin admin] PATH_INFO: `/v3/users/782118db40e84d38897c252d252d11c9` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:13:52.054029 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4137b9-03d1-422c-af07-361998d2c296 admin admin] RBAC: Authorizing `identity:delete_user(user_id=782118db40e84d38897c252d252d11c9)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:13:52.055971 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cf4137b9-03d1-422c-af07-361998d2c296 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:13:52.069250 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 999/2002] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jul 13 22:13:51 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 22:13:52.069700 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:13:52.077552 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a292a3bd-0ca5-4300-959d-2a1decb08595 None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:13:52.077843 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a292a3bd-0ca5-4300-959d-2a1decb08595 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:13:52.078124 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a292a3bd-0ca5-4300-959d-2a1decb08595 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:13:52.097208 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-a292a3bd-0ca5-4300-959d-2a1decb08595 None None] MFA Rules not processed for user `cc99c93da5074cf5a48bc2aa9a92c5e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 22:13:52.098424 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-a292a3bd-0ca5-4300-959d-2a1decb08595 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:13:52.105338 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-cf4137b9-03d1-422c-af07-361998d2c296 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '782118db40e84d38897c252d252d11c9'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:13:52.106480 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1000/2003] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jul 13 22:13:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 13 22:13:52.110853 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a292a3bd-0ca5-4300-959d-2a1decb08595 None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:13:52.113775 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-cf4137b9-03d1-422c-af07-361998d2c296 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '782118db40e84d38897c252d252d11c9'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:13:52.120402 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a292a3bd-0ca5-4300-959d-2a1decb08595 None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:13:52.125324 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4c2fbed3-e5be-47b2-aca9-ac2236118549 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:13:52.126024 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-cf4137b9-03d1-422c-af07-361998d2c296 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '782118db40e84d38897c252d252d11c9'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:13:52.126170 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2fbed3-e5be-47b2-aca9-ac2236118549 admin admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:13:52.126392 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2fbed3-e5be-47b2-aca9-ac2236118549 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:13:52.126721 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4c2fbed3-e5be-47b2-aca9-ac2236118549 admin admin] PATH_INFO: `/v3/domains` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:13:52.126829 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-cf4137b9-03d1-422c-af07-361998d2c296 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '782118db40e84d38897c252d252d11c9'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:13:52.127357 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-cf4137b9-03d1-422c-af07-361998d2c296 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '782118db40e84d38897c252d252d11c9'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:13:52.127508 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2fbed3-e5be-47b2-aca9-ac2236118549 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 22:13:52.127772 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2fbed3-e5be-47b2-aca9-ac2236118549 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:13:52.129566 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4c2fbed3-e5be-47b2-aca9-ac2236118549 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:13:52.139665 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1004/2004] 10.222.0.52 () {58 vars in 1223 bytes} [Wed Jul 13 22:13:51 2022] DELETE /identity/v3/users/782118db40e84d38897c252d252d11c9 => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 13 22:13:52.144819 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1001/2005] 10.222.0.52 () {58 vars in 1152 bytes} [Wed Jul 13 22:13:52 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 22:13:52.146323 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4137b9-03d1-422c-af07-361998d2c296 admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:13:52.146780 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c57425-f2fd-4659-9f24-802ccf6c0185 None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:13:52.147070 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c57425-f2fd-4659-9f24-802ccf6c0185 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:13:52.147344 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b1c57425-f2fd-4659-9f24-802ccf6c0185 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:13:52.147822 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-cf4137b9-03d1-422c-af07-361998d2c296 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:13:52.166270 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-b1c57425-f2fd-4659-9f24-802ccf6c0185 None None] MFA Rules not processed for user `cc99c93da5074cf5a48bc2aa9a92c5e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 22:13:52.167480 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-b1c57425-f2fd-4659-9f24-802ccf6c0185 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:13:52.214281 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cf4137b9-03d1-422c-af07-361998d2c296 admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:13:52.219131 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffeca3e5-9625-4e83-b992-da9e8648e4d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:13:52.219783 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffeca3e5-9625-4e83-b992-da9e8648e4d3 admin admin] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:13:52.220060 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffeca3e5-9625-4e83-b992-da9e8648e4d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:13:52.220333 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffeca3e5-9625-4e83-b992-da9e8648e4d3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:13:52.221077 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffeca3e5-9625-4e83-b992-da9e8648e4d3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:13:52.223267 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffeca3e5-9625-4e83-b992-da9e8648e4d3 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:13:52.226781 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1002/2006] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jul 13 22:13:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 81 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 22:13:52.237472 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c57425-f2fd-4659-9f24-802ccf6c0185 None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:13:52.239094 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-b1c57425-f2fd-4659-9f24-802ccf6c0185 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:13:52.252706 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1005/2007] 10.222.0.52 () {60 vars in 1152 bytes} [Wed Jul 13 22:13:52 2022] POST /identity/v3/projects => generated 381 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 22:13:52.253702 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffeca3e5-9625-4e83-b992-da9e8648e4d3 admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:13:52.297140 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b1c57425-f2fd-4659-9f24-802ccf6c0185 None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:13:52.318476 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffeca3e5-9625-4e83-b992-da9e8648e4d3 admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:13:52.323579 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dbca1d3-a7f0-4b30-8abb-703c719c5f9f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:13:52.324250 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbca1d3-a7f0-4b30-8abb-703c719c5f9f admin admin] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:13:52.324537 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbca1d3-a7f0-4b30-8abb-703c719c5f9f admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:13:52.325115 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4dbca1d3-a7f0-4b30-8abb-703c719c5f9f admin admin] PATH_INFO: `/v3/users` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:13:52.325491 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dbca1d3-a7f0-4b30-8abb-703c719c5f9f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:13:52.327477 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4dbca1d3-a7f0-4b30-8abb-703c719c5f9f admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:13:52.354100 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1006/2008] 10.222.0.52 () {60 vars in 1143 bytes} [Wed Jul 13 22:13:52 2022] POST /identity/v3/users => generated 291 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 13 22:13:52.358377 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dbca1d3-a7f0-4b30-8abb-703c719c5f9f admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:13:52.365521 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2613d20-bf80-4864-9c74-5a7879d87523 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '847ec9a1bd4d4dec804318a5c845895a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417e3e2f1333481795a089bef2e24bea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:13:52.366199 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2613d20-bf80-4864-9c74-5a7879d87523 service neutron] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:13:52.366503 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2613d20-bf80-4864-9c74-5a7879d87523 service neutron] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:13:52.366805 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b2613d20-bf80-4864-9c74-5a7879d87523 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:13:52.372940 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4dbca1d3-a7f0-4b30-8abb-703c719c5f9f admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:13:52.379118 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e3edd5-6d4d-494d-892e-67ce0b694ec0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:13:52.379749 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e3edd5-6d4d-494d-892e-67ce0b694ec0 admin admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:13:52.379866 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2613d20-bf80-4864-9c74-5a7879d87523 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:13:52.380123 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e3edd5-6d4d-494d-892e-67ce0b694ec0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:13:52.381151 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21e3edd5-6d4d-494d-892e-67ce0b694ec0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:13:52.381151 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e3edd5-6d4d-494d-892e-67ce0b694ec0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:13:52.381731 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b2613d20-bf80-4864-9c74-5a7879d87523 service neutron] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:13:52.382978 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21e3edd5-6d4d-494d-892e-67ce0b694ec0 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:13:52.384615 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=86,keystone:DELETE=9,keystone:INSERT=5 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:13:52.393510 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1003/2009] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jul 13 22:13:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 157 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 22:13:52.400000 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1007/2010] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jul 13 22:13:52 2022] GET /identity/v3/roles => generated 1420 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 22:13:52.403616 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2613d20-bf80-4864-9c74-5a7879d87523 service neutron] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:13:52.413346 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b2613d20-bf80-4864-9c74-5a7879d87523 service neutron] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:13:52.417994 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb0ba77a-e303-4744-bb83-c15da348ad73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:13:52.418693 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0ba77a-e303-4744-bb83-c15da348ad73 admin admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:13:52.418977 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0ba77a-e303-4744-bb83-c15da348ad73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:13:52.419250 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb0ba77a-e303-4744-bb83-c15da348ad73 admin admin] PATH_INFO: `/v3/roles` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:13:52.419984 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0ba77a-e303-4744-bb83-c15da348ad73 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:13:52.421598 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb0ba77a-e303-4744-bb83-c15da348ad73 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:13:52.432202 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1004/2011] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jul 13 22:13:52 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 22:13:52.436270 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e3edd5-6d4d-494d-892e-67ce0b694ec0 admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:13:52.446910 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21e3edd5-6d4d-494d-892e-67ce0b694ec0 admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:13:52.451917 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56eb2eb7-6f91-415f-aba5-7397f5c11443 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:13:52.452818 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56eb2eb7-6f91-415f-aba5-7397f5c11443 admin admin] REQUEST_METHOD: `PUT` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:13:52.453126 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56eb2eb7-6f91-415f-aba5-7397f5c11443 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:13:52.453403 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56eb2eb7-6f91-415f-aba5-7397f5c11443 admin admin] PATH_INFO: `/v3/projects/6587cfa2404644d5bf2100f2a49db1aa/users/e46d01dd355c4f0b9d175d139c285e3f/roles/2a5e86976394498abb6310697cdb3f8b` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:13:52.470447 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56eb2eb7-6f91-415f-aba5-7397f5c11443 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6587cfa2404644d5bf2100f2a49db1aa, user_id=e46d01dd355c4f0b9d175d139c285e3f, role_id=2a5e86976394498abb6310697cdb3f8b)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:13:52.473190 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56eb2eb7-6f91-415f-aba5-7397f5c11443 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:13:52.489070 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1008/2012] 10.222.0.52 () {60 vars in 1482 bytes} [Wed Jul 13 22:13:52 2022] PUT /identity/v3/projects/6587cfa2404644d5bf2100f2a49db1aa/users/e46d01dd355c4f0b9d175d139c285e3f/roles/2a5e86976394498abb6310697cdb3f8b => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 22:13:52.495174 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7141f094-502b-404f-be7b-6368bbe571ae None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:13:52.495504 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7141f094-502b-404f-be7b-6368bbe571ae None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:13:52.495767 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7141f094-502b-404f-be7b-6368bbe571ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:13:52.514760 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-7141f094-502b-404f-be7b-6368bbe571ae None None] MFA Rules not processed for user `cc99c93da5074cf5a48bc2aa9a92c5e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 22:13:52.516004 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-7141f094-502b-404f-be7b-6368bbe571ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:13:52.520221 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:13:52.589850 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1005/2013] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jul 13 22:13:52 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 22:13:52.605013 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56eb2eb7-6f91-415f-aba5-7397f5c11443 admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:13:52.662269 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56eb2eb7-6f91-415f-aba5-7397f5c11443 admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:13:52.666320 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99565119-a06e-452c-a735-cb18be07ecb1 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '847ec9a1bd4d4dec804318a5c845895a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417e3e2f1333481795a089bef2e24bea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:13:52.667110 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99565119-a06e-452c-a735-cb18be07ecb1 service neutron] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:13:52.667427 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99565119-a06e-452c-a735-cb18be07ecb1 service neutron] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:13:52.667701 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-99565119-a06e-452c-a735-cb18be07ecb1 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:13:52.677639 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99565119-a06e-452c-a735-cb18be07ecb1 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:13:52.679142 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-99565119-a06e-452c-a735-cb18be07ecb1 service neutron] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:13:52.689567 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1009/2014] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jul 13 22:13:52 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 22:13:52.737174 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7141f094-502b-404f-be7b-6368bbe571ae None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:13:52.738893 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-7141f094-502b-404f-be7b-6368bbe571ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:13:52.754027 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7141f094-502b-404f-be7b-6368bbe571ae None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:13:52.760568 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-12a6e63f-1ba1-47a9-b28e-c3cd7827ffbd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:13:52.761469 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a6e63f-1ba1-47a9-b28e-c3cd7827ffbd admin admin] REQUEST_METHOD: `DELETE` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:13:52.761744 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a6e63f-1ba1-47a9-b28e-c3cd7827ffbd admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:13:52.762015 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-12a6e63f-1ba1-47a9-b28e-c3cd7827ffbd admin admin] PATH_INFO: `/v3/projects/ea4bb4885ad0471c8d41785429f3cded` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:13:52.763943 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a6e63f-1ba1-47a9-b28e-c3cd7827ffbd admin admin] RBAC: Authorizing `identity:delete_project(project_id=ea4bb4885ad0471c8d41785429f3cded)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:13:52.766025 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-12a6e63f-1ba1-47a9-b28e-c3cd7827ffbd admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:13:52.821273 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-12a6e63f-1ba1-47a9-b28e-c3cd7827ffbd admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ea4bb4885ad0471c8d41785429f3cded', 'request_id': 'req-12a6e63f-1ba1-47a9-b28e-c3cd7827ffbd'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:13:52.828821 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-12a6e63f-1ba1-47a9-b28e-c3cd7827ffbd admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ea4bb4885ad0471c8d41785429f3cded', 'request_id': 'req-12a6e63f-1ba1-47a9-b28e-c3cd7827ffbd'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:13:52.836268 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-12a6e63f-1ba1-47a9-b28e-c3cd7827ffbd admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ea4bb4885ad0471c8d41785429f3cded', 'request_id': 'req-12a6e63f-1ba1-47a9-b28e-c3cd7827ffbd'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:13:52.849951 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-12a6e63f-1ba1-47a9-b28e-c3cd7827ffbd admin admin] The token cache is being invalidate because project ea4bb4885ad0471c8d41785429f3cded was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80382) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 22:13:52.850241 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-12a6e63f-1ba1-47a9-b28e-c3cd7827ffbd admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:13:52.850893 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-12a6e63f-1ba1-47a9-b28e-c3cd7827ffbd admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:13:52.852739 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1006/2015] 10.222.0.52 () {58 vars in 1232 bytes} [Wed Jul 13 22:13:52 2022] DELETE /identity/v3/projects/ea4bb4885ad0471c8d41785429f3cded => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 22:13:52.856676 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99565119-a06e-452c-a735-cb18be07ecb1 service neutron] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:13:52.866685 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-99565119-a06e-452c-a735-cb18be07ecb1 service neutron] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:13:52.932849 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9d51b3-a8b7-45d7-b3fe-b2fbd31d5614 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:13:52.933908 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9d51b3-a8b7-45d7-b3fe-b2fbd31d5614 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:13:52.934061 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9d51b3-a8b7-45d7-b3fe-b2fbd31d5614 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:13:52.934365 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e9d51b3-a8b7-45d7-b3fe-b2fbd31d5614 admin admin] PATH_INFO: `/v3/users/818166c1132d4011bfeb907555d20456` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:13:52.935652 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9d51b3-a8b7-45d7-b3fe-b2fbd31d5614 admin admin] RBAC: Authorizing `identity:delete_user(user_id=818166c1132d4011bfeb907555d20456)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:13:52.937724 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e9d51b3-a8b7-45d7-b3fe-b2fbd31d5614 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:13:53.005054 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-8e9d51b3-a8b7-45d7-b3fe-b2fbd31d5614 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '818166c1132d4011bfeb907555d20456'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:13:53.024675 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-8e9d51b3-a8b7-45d7-b3fe-b2fbd31d5614 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '818166c1132d4011bfeb907555d20456'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:13:53.044628 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-8e9d51b3-a8b7-45d7-b3fe-b2fbd31d5614 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '818166c1132d4011bfeb907555d20456'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:13:53.045575 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-8e9d51b3-a8b7-45d7-b3fe-b2fbd31d5614 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '818166c1132d4011bfeb907555d20456'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:13:53.046200 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-8e9d51b3-a8b7-45d7-b3fe-b2fbd31d5614 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '818166c1132d4011bfeb907555d20456'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:13:53.052433 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:13:53.065684 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1010/2016] 10.222.0.52 () {58 vars in 1223 bytes} [Wed Jul 13 22:13:52 2022] DELETE /identity/v3/users/818166c1132d4011bfeb907555d20456 => generated 0 bytes in 209 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 13 22:13:53.480786 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa52d1d8-39b6-4e52-8d74-8502392187d1 None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:13:53.481391 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa52d1d8-39b6-4e52-8d74-8502392187d1 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:13:53.482242 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa52d1d8-39b6-4e52-8d74-8502392187d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:13:53.500998 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9d51b3-a8b7-45d7-b3fe-b2fbd31d5614 admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:13:53.502722 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-8e9d51b3-a8b7-45d7-b3fe-b2fbd31d5614 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:13:53.532928 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-aa52d1d8-39b6-4e52-8d74-8502392187d1 None None] MFA Rules not processed for user `e46d01dd355c4f0b9d175d139c285e3f`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 22:13:53.534272 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-aa52d1d8-39b6-4e52-8d74-8502392187d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:13:53.597822 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e9d51b3-a8b7-45d7-b3fe-b2fbd31d5614 admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:13:53.606805 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-470e20d9-7d88-4e32-b440-8f6f1a764504 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:13:53.609176 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470e20d9-7d88-4e32-b440-8f6f1a764504 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:13:53.609582 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470e20d9-7d88-4e32-b440-8f6f1a764504 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:13:53.609854 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-470e20d9-7d88-4e32-b440-8f6f1a764504 admin admin] PATH_INFO: `/v3/projects/8514586aa53141629da48ce21495e6e4` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:13:53.613051 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470e20d9-7d88-4e32-b440-8f6f1a764504 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8514586aa53141629da48ce21495e6e4)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:13:53.615065 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-470e20d9-7d88-4e32-b440-8f6f1a764504 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:13:53.644971 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1007/2017] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jul 13 22:13:53 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 166 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 22:13:53.662323 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa52d1d8-39b6-4e52-8d74-8502392187d1 None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:13:53.663749 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-aa52d1d8-39b6-4e52-8d74-8502392187d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:13:53.682388 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-470e20d9-7d88-4e32-b440-8f6f1a764504 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8514586aa53141629da48ce21495e6e4', 'request_id': 'req-470e20d9-7d88-4e32-b440-8f6f1a764504'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:13:53.691149 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-470e20d9-7d88-4e32-b440-8f6f1a764504 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8514586aa53141629da48ce21495e6e4', 'request_id': 'req-470e20d9-7d88-4e32-b440-8f6f1a764504'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:13:53.695071 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:13:53.699249 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-470e20d9-7d88-4e32-b440-8f6f1a764504 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8514586aa53141629da48ce21495e6e4', 'request_id': 'req-470e20d9-7d88-4e32-b440-8f6f1a764504'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:13:53.715803 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-470e20d9-7d88-4e32-b440-8f6f1a764504 admin admin] The token cache is being invalidate because project 8514586aa53141629da48ce21495e6e4 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80381) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 22:13:53.716097 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-470e20d9-7d88-4e32-b440-8f6f1a764504 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:13:53.716766 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-470e20d9-7d88-4e32-b440-8f6f1a764504 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:13:53.718327 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1011/2018] 10.222.0.52 () {58 vars in 1232 bytes} [Wed Jul 13 22:13:53 2022] DELETE /identity/v3/projects/8514586aa53141629da48ce21495e6e4 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 13 22:13:53.724493 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2898a9a3-f220-4fa5-b1ef-540139a088ee None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:13:53.724779 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2898a9a3-f220-4fa5-b1ef-540139a088ee None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:13:53.725053 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2898a9a3-f220-4fa5-b1ef-540139a088ee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:13:53.737755 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa52d1d8-39b6-4e52-8d74-8502392187d1 None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:13:53.744501 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-2898a9a3-f220-4fa5-b1ef-540139a088ee None None] MFA Rules not processed for user `cc99c93da5074cf5a48bc2aa9a92c5e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 22:13:53.745922 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-2898a9a3-f220-4fa5-b1ef-540139a088ee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:13:53.787869 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f462b66-2c0f-4b69-a626-72400df0b010 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc77e1ea1ada4700b809beb1c980c5e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417e3e2f1333481795a089bef2e24bea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:13:53.788416 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f462b66-2c0f-4b69-a626-72400df0b010 service cinder] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:13:53.788693 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f462b66-2c0f-4b69-a626-72400df0b010 service cinder] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:13:53.788960 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f462b66-2c0f-4b69-a626-72400df0b010 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:13:53.790783 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-9f462b66-2c0f-4b69-a626-72400df0b010 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:13:53.805223 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1012/2019] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jul 13 22:13:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 83 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 13 22:13:53.813335 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cd0bcb-6531-44e1-bd90-d1b65ba21bca None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:13:53.813619 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cd0bcb-6531-44e1-bd90-d1b65ba21bca None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:13:53.813920 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c0cd0bcb-6531-44e1-bd90-d1b65ba21bca None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:13:53.832617 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-c0cd0bcb-6531-44e1-bd90-d1b65ba21bca None None] MFA Rules not processed for user `cc99c93da5074cf5a48bc2aa9a92c5e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 22:13:53.833806 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-c0cd0bcb-6531-44e1-bd90-d1b65ba21bca None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:13:53.841716 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1013/2020] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jul 13 22:13:53 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 13 22:13:53.845787 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0cd0bcb-6531-44e1-bd90-d1b65ba21bca None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:13:53.851452 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f462b66-2c0f-4b69-a626-72400df0b010 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:13:53.852529 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f462b66-2c0f-4b69-a626-72400df0b010 service cinder] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:13:53.854685 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c0cd0bcb-6531-44e1-bd90-d1b65ba21bca None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:13:53.859276 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1520b2-a50e-4125-8d6c-0f148593b5ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:13:53.859914 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1520b2-a50e-4125-8d6c-0f148593b5ea admin admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:13:53.860193 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1520b2-a50e-4125-8d6c-0f148593b5ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:13:53.860464 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e1520b2-a50e-4125-8d6c-0f148593b5ea admin admin] PATH_INFO: `/v3/domains` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:13:53.861179 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1520b2-a50e-4125-8d6c-0f148593b5ea admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 22:13:53.861479 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1520b2-a50e-4125-8d6c-0f148593b5ea admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:13:53.861939 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1008/2021] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jul 13 22:13:53 2022] GET /identity/v3/auth/tokens => generated 3248 bytes in 200 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 13 22:13:53.863174 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e1520b2-a50e-4125-8d6c-0f148593b5ea admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:13:53.878575 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1014/2022] 10.222.0.52 () {58 vars in 1152 bytes} [Wed Jul 13 22:13:53 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 22:13:53.879414 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f462b66-2c0f-4b69-a626-72400df0b010 service cinder] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:13:53.890594 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f462b66-2c0f-4b69-a626-72400df0b010 service cinder] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:13:53.895381 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4eff1a-63ec-41f4-b50b-40d5957b049c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:13:53.896201 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4eff1a-63ec-41f4-b50b-40d5957b049c admin admin] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:13:53.896482 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4eff1a-63ec-41f4-b50b-40d5957b049c admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:13:53.896751 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4eff1a-63ec-41f4-b50b-40d5957b049c admin admin] PATH_INFO: `/v3/projects` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:13:53.897446 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4eff1a-63ec-41f4-b50b-40d5957b049c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:13:53.899420 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4eff1a-63ec-41f4-b50b-40d5957b049c admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:13:53.928006 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1009/2023] 10.222.0.52 () {60 vars in 1152 bytes} [Wed Jul 13 22:13:53 2022] POST /identity/v3/projects => generated 379 bytes in 49 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 22:13:53.932424 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1520b2-a50e-4125-8d6c-0f148593b5ea admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:13:53.938733 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:13:53.943108 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e1520b2-a50e-4125-8d6c-0f148593b5ea admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:13:54.009039 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00770c0e-e38e-4c2c-834e-ff65966493c3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:13:54.010144 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00770c0e-e38e-4c2c-834e-ff65966493c3 admin admin] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:13:54.010144 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00770c0e-e38e-4c2c-834e-ff65966493c3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:13:54.010584 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-00770c0e-e38e-4c2c-834e-ff65966493c3 admin admin] PATH_INFO: `/v3/users` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:13:54.011386 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00770c0e-e38e-4c2c-834e-ff65966493c3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:13:54.013430 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-00770c0e-e38e-4c2c-834e-ff65966493c3 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:13:54.037409 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1015/2024] 10.222.0.52 () {60 vars in 1143 bytes} [Wed Jul 13 22:13:53 2022] POST /identity/v3/users => generated 290 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 22:13:54.041750 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4eff1a-63ec-41f4-b50b-40d5957b049c admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:13:54.053052 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4eff1a-63ec-41f4-b50b-40d5957b049c admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:13:54.058159 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6578375-fc8c-4d1e-b483-87bfea854c20 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:13:54.059001 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6578375-fc8c-4d1e-b483-87bfea854c20 admin admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:13:54.059347 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6578375-fc8c-4d1e-b483-87bfea854c20 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:13:54.059686 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6578375-fc8c-4d1e-b483-87bfea854c20 admin admin] PATH_INFO: `/v3/roles` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:13:54.060445 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6578375-fc8c-4d1e-b483-87bfea854c20 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:13:54.062052 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6578375-fc8c-4d1e-b483-87bfea854c20 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:13:54.075412 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1010/2025] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jul 13 22:13:54 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 22:13:54.079575 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00770c0e-e38e-4c2c-834e-ff65966493c3 admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:13:54.089593 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-00770c0e-e38e-4c2c-834e-ff65966493c3 admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:13:54.095109 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-331f2969-26fb-4d36-aff1-49d1653ddd79 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:13:54.095781 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331f2969-26fb-4d36-aff1-49d1653ddd79 admin admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:13:54.096135 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331f2969-26fb-4d36-aff1-49d1653ddd79 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:13:54.096415 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-331f2969-26fb-4d36-aff1-49d1653ddd79 admin admin] PATH_INFO: `/v3/roles` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:13:54.097340 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-331f2969-26fb-4d36-aff1-49d1653ddd79 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:13:54.099198 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-331f2969-26fb-4d36-aff1-49d1653ddd79 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:13:54.109711 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1016/2026] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jul 13 22:13:54 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 13 22:13:54.114546 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6578375-fc8c-4d1e-b483-87bfea854c20 admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:13:54.124898 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6578375-fc8c-4d1e-b483-87bfea854c20 admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:13:54.129853 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358dadb2-d617-4514-8117-fa4fee875298 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:13:54.130788 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358dadb2-d617-4514-8117-fa4fee875298 admin admin] REQUEST_METHOD: `PUT` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:13:54.131072 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358dadb2-d617-4514-8117-fa4fee875298 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:13:54.131372 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-358dadb2-d617-4514-8117-fa4fee875298 admin admin] PATH_INFO: `/v3/projects/8ab45edd2d124f32817853049c50e467/users/ce9297274fe84e13a446c3b25e1a29bb/roles/2a5e86976394498abb6310697cdb3f8b` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:13:54.148676 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-358dadb2-d617-4514-8117-fa4fee875298 admin admin] RBAC: Authorizing `identity:create_grant(project_id=8ab45edd2d124f32817853049c50e467, user_id=ce9297274fe84e13a446c3b25e1a29bb, role_id=2a5e86976394498abb6310697cdb3f8b)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:13:54.151462 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-358dadb2-d617-4514-8117-fa4fee875298 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:13:54.166286 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1011/2027] 10.222.0.52 () {60 vars in 1482 bytes} [Wed Jul 13 22:13:54 2022] PUT /identity/v3/projects/8ab45edd2d124f32817853049c50e467/users/ce9297274fe84e13a446c3b25e1a29bb/roles/2a5e86976394498abb6310697cdb3f8b => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 22:13:54.178651 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb2a801-b859-4984-aca4-99636408bba5 None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:13:54.179133 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb2a801-b859-4984-aca4-99636408bba5 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:13:54.179433 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9cb2a801-b859-4984-aca4-99636408bba5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:13:54.209876 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-9cb2a801-b859-4984-aca4-99636408bba5 None None] MFA Rules not processed for user `ce9297274fe84e13a446c3b25e1a29bb`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 22:13:54.211262 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-9cb2a801-b859-4984-aca4-99636408bba5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:13:54.280697 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1017/2028] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jul 13 22:13:54 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 104 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 22:13:54.285454 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358dadb2-d617-4514-8117-fa4fee875298 admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:13:54.325466 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:13:54.351949 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-358dadb2-d617-4514-8117-fa4fee875298 admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:13:54.356984 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f71aec-a1ac-4db9-8f0d-b8a8395abb72 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:13:54.357653 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f71aec-a1ac-4db9-8f0d-b8a8395abb72 admin admin] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:13:54.357925 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f71aec-a1ac-4db9-8f0d-b8a8395abb72 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:13:54.358190 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b3f71aec-a1ac-4db9-8f0d-b8a8395abb72 admin admin] PATH_INFO: `/v3/projects` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:13:54.358919 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f71aec-a1ac-4db9-8f0d-b8a8395abb72 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:13:54.360865 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b3f71aec-a1ac-4db9-8f0d-b8a8395abb72 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:13:54.390194 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1012/2029] 10.222.0.52 () {60 vars in 1152 bytes} [Wed Jul 13 22:13:54 2022] POST /identity/v3/projects => generated 383 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 22:13:54.394512 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb2a801-b859-4984-aca4-99636408bba5 None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:13:54.405805 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9cb2a801-b859-4984-aca4-99636408bba5 None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:13:54.470146 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a4e02b-2e28-4351-9521-8543f4298e83 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:13:54.470868 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a4e02b-2e28-4351-9521-8543f4298e83 admin admin] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:13:54.471384 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a4e02b-2e28-4351-9521-8543f4298e83 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:13:54.471664 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b8a4e02b-2e28-4351-9521-8543f4298e83 admin admin] PATH_INFO: `/v3/users` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:13:54.472375 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a4e02b-2e28-4351-9521-8543f4298e83 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:13:54.474336 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b8a4e02b-2e28-4351-9521-8543f4298e83 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:13:54.495809 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1018/2030] 10.222.0.52 () {60 vars in 1143 bytes} [Wed Jul 13 22:13:54 2022] POST /identity/v3/users => generated 298 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 22:13:54.499876 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f71aec-a1ac-4db9-8f0d-b8a8395abb72 admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:13:54.510654 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b3f71aec-a1ac-4db9-8f0d-b8a8395abb72 admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:13:54.515458 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ac4a40-fb4e-4abb-bd87-01e6d13287b8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:13:54.516125 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ac4a40-fb4e-4abb-bd87-01e6d13287b8 admin admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:13:54.516408 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ac4a40-fb4e-4abb-bd87-01e6d13287b8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:13:54.516685 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47ac4a40-fb4e-4abb-bd87-01e6d13287b8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:13:54.517426 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ac4a40-fb4e-4abb-bd87-01e6d13287b8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:13:54.519111 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47ac4a40-fb4e-4abb-bd87-01e6d13287b8 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:13:54.530010 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1013/2031] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jul 13 22:13:54 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 22:13:54.534309 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a4e02b-2e28-4351-9521-8543f4298e83 admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:13:54.539841 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:13:54.546138 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b8a4e02b-2e28-4351-9521-8543f4298e83 admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:13:54.552934 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7593285f-6977-4cf1-a810-593969852121 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:13:54.553774 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7593285f-6977-4cf1-a810-593969852121 admin admin] REQUEST_METHOD: `PUT` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:13:54.554051 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7593285f-6977-4cf1-a810-593969852121 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:13:54.554321 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7593285f-6977-4cf1-a810-593969852121 admin admin] PATH_INFO: `/v3/projects/2a2b018a2a324f78b5f47ddbc73983fd/users/0de019a339ec4b0dae68f86c6358c6fe/roles/44efba2579f342df8a0789c59d163ec3` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:13:54.573474 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7593285f-6977-4cf1-a810-593969852121 admin admin] RBAC: Authorizing `identity:create_grant(project_id=2a2b018a2a324f78b5f47ddbc73983fd, user_id=0de019a339ec4b0dae68f86c6358c6fe, role_id=44efba2579f342df8a0789c59d163ec3)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:13:54.576245 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7593285f-6977-4cf1-a810-593969852121 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:13:54.592499 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1019/2032] 10.222.0.52 () {60 vars in 1482 bytes} [Wed Jul 13 22:13:54 2022] PUT /identity/v3/projects/2a2b018a2a324f78b5f47ddbc73983fd/users/0de019a339ec4b0dae68f86c6358c6fe/roles/44efba2579f342df8a0789c59d163ec3 => generated 0 bytes in 59 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 22:13:54.597189 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ac4a40-fb4e-4abb-bd87-01e6d13287b8 admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:13:54.612932 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47ac4a40-fb4e-4abb-bd87-01e6d13287b8 admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:13:54.678351 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c53c3597-fee0-464a-838d-cd236ede7180 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:13:54.679210 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53c3597-fee0-464a-838d-cd236ede7180 admin admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:13:54.679468 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53c3597-fee0-464a-838d-cd236ede7180 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:13:54.679767 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c53c3597-fee0-464a-838d-cd236ede7180 admin admin] PATH_INFO: `/v3/roles` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:13:54.680583 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53c3597-fee0-464a-838d-cd236ede7180 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:13:54.682214 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c53c3597-fee0-464a-838d-cd236ede7180 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:13:54.694320 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1014/2033] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jul 13 22:13:54 2022] GET /identity/v3/roles => generated 1420 bytes in 98 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 13 22:13:54.700070 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7593285f-6977-4cf1-a810-593969852121 admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:13:54.712798 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7593285f-6977-4cf1-a810-593969852121 admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:13:54.719483 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583cd9b9-a26b-454c-ad55-0baa19dcecfa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:13:54.720669 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583cd9b9-a26b-454c-ad55-0baa19dcecfa admin admin] REQUEST_METHOD: `PUT` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:13:54.720956 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583cd9b9-a26b-454c-ad55-0baa19dcecfa admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:13:54.721288 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-583cd9b9-a26b-454c-ad55-0baa19dcecfa admin admin] PATH_INFO: `/v3/projects/2a2b018a2a324f78b5f47ddbc73983fd/users/0de019a339ec4b0dae68f86c6358c6fe/roles/44efba2579f342df8a0789c59d163ec3` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:13:54.724878 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-583cd9b9-a26b-454c-ad55-0baa19dcecfa admin admin] RBAC: Authorizing `identity:create_grant(project_id=2a2b018a2a324f78b5f47ddbc73983fd, user_id=0de019a339ec4b0dae68f86c6358c6fe, role_id=44efba2579f342df8a0789c59d163ec3)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:13:54.727489 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-583cd9b9-a26b-454c-ad55-0baa19dcecfa admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:13:54.750612 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1020/2034] 10.222.0.52 () {60 vars in 1482 bytes} [Wed Jul 13 22:13:54 2022] PUT /identity/v3/projects/2a2b018a2a324f78b5f47ddbc73983fd/users/0de019a339ec4b0dae68f86c6358c6fe/roles/44efba2579f342df8a0789c59d163ec3 => generated 0 bytes in 42 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 22:13:54.756696 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d8480b-e8a3-4214-a873-45a2cde7139f None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:13:54.756988 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d8480b-e8a3-4214-a873-45a2cde7139f None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:13:54.757274 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1d8480b-e8a3-4214-a873-45a2cde7139f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:13:54.789706 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-d1d8480b-e8a3-4214-a873-45a2cde7139f None None] MFA Rules not processed for user `0de019a339ec4b0dae68f86c6358c6fe`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 22:13:54.791183 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-d1d8480b-e8a3-4214-a873-45a2cde7139f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:13:54.862837 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1015/2035] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jul 13 22:13:54 2022] POST /identity/v3/auth/tokens => generated 3317 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 22:13:54.873998 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583cd9b9-a26b-454c-ad55-0baa19dcecfa admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:13:54.876009 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-583cd9b9-a26b-454c-ad55-0baa19dcecfa admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:13:54.938651 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-583cd9b9-a26b-454c-ad55-0baa19dcecfa admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:13:54.942654 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-952c66d7-6395-4a88-95d3-5d6c7f0a273a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc77e1ea1ada4700b809beb1c980c5e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417e3e2f1333481795a089bef2e24bea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:13:54.943415 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952c66d7-6395-4a88-95d3-5d6c7f0a273a service cinder] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:13:54.943689 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952c66d7-6395-4a88-95d3-5d6c7f0a273a service cinder] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:13:54.943956 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-952c66d7-6395-4a88-95d3-5d6c7f0a273a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:13:54.954118 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952c66d7-6395-4a88-95d3-5d6c7f0a273a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:13:54.955785 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-952c66d7-6395-4a88-95d3-5d6c7f0a273a service cinder] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:13:54.966180 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1021/2036] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jul 13 22:13:54 2022] GET /identity/v3/auth/tokens => generated 3317 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 22:13:55.068368 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d8480b-e8a3-4214-a873-45a2cde7139f None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:13:55.079441 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1d8480b-e8a3-4214-a873-45a2cde7139f None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:13:55.084993 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67fc2a1b-f51d-4ab7-bcf1-47fbc4205ddc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc77e1ea1ada4700b809beb1c980c5e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417e3e2f1333481795a089bef2e24bea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:13:55.085761 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fc2a1b-f51d-4ab7-bcf1-47fbc4205ddc service cinder] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:13:55.086039 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fc2a1b-f51d-4ab7-bcf1-47fbc4205ddc service cinder] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:13:55.086313 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-67fc2a1b-f51d-4ab7-bcf1-47fbc4205ddc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:13:55.118293 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:13:55.147061 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67fc2a1b-f51d-4ab7-bcf1-47fbc4205ddc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:13:55.148521 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-67fc2a1b-f51d-4ab7-bcf1-47fbc4205ddc service cinder] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:13:55.158006 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1016/2037] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jul 13 22:13:55 2022] GET /identity/v3/auth/tokens => generated 3246 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 22:13:56.326392 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe246de-747e-4261-983f-132d16c8e2cb None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:13:56.327406 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe246de-747e-4261-983f-132d16c8e2cb None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:13:56.327683 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fe246de-747e-4261-983f-132d16c8e2cb None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:13:56.347517 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-2fe246de-747e-4261-983f-132d16c8e2cb None None] MFA Rules not processed for user `e46d01dd355c4f0b9d175d139c285e3f`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 22:13:56.348743 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-2fe246de-747e-4261-983f-132d16c8e2cb None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:13:56.407612 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1022/2038] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jul 13 22:13:56 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 13 22:13:57.810972 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67fc2a1b-f51d-4ab7-bcf1-47fbc4205ddc service cinder] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:13:57.812483 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-67fc2a1b-f51d-4ab7-bcf1-47fbc4205ddc service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:13:57.925055 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-67fc2a1b-f51d-4ab7-bcf1-47fbc4205ddc service cinder] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:13:57.929871 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729bf3c4-14e2-45df-b1d6-d00d76cf8da5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:13:57.931129 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729bf3c4-14e2-45df-b1d6-d00d76cf8da5 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:13:57.931401 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729bf3c4-14e2-45df-b1d6-d00d76cf8da5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:13:57.931901 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-729bf3c4-14e2-45df-b1d6-d00d76cf8da5 admin admin] PATH_INFO: `/v3/users/1bbc07a53d5d48af921e0aebe00d6da7` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:13:57.933354 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729bf3c4-14e2-45df-b1d6-d00d76cf8da5 admin admin] RBAC: Authorizing `identity:delete_user(user_id=1bbc07a53d5d48af921e0aebe00d6da7)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:13:57.935793 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-729bf3c4-14e2-45df-b1d6-d00d76cf8da5 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:13:57.987090 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-729bf3c4-14e2-45df-b1d6-d00d76cf8da5 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1bbc07a53d5d48af921e0aebe00d6da7'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:13:57.995641 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-729bf3c4-14e2-45df-b1d6-d00d76cf8da5 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1bbc07a53d5d48af921e0aebe00d6da7'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:13:58.007377 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-729bf3c4-14e2-45df-b1d6-d00d76cf8da5 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1bbc07a53d5d48af921e0aebe00d6da7'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:13:58.008003 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-729bf3c4-14e2-45df-b1d6-d00d76cf8da5 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1bbc07a53d5d48af921e0aebe00d6da7'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:13:58.008616 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-729bf3c4-14e2-45df-b1d6-d00d76cf8da5 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1bbc07a53d5d48af921e0aebe00d6da7'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:13:58.020681 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1017/2039] 10.222.0.52 () {58 vars in 1223 bytes} [Wed Jul 13 22:13:57 2022] DELETE /identity/v3/users/1bbc07a53d5d48af921e0aebe00d6da7 => generated 0 bytes in 210 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 22:13:58.025600 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9264eaae-f812-4c0d-95b7-de37458d63bf None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:13:58.026091 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9264eaae-f812-4c0d-95b7-de37458d63bf None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:13:58.026182 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9264eaae-f812-4c0d-95b7-de37458d63bf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:13:58.045740 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-9264eaae-f812-4c0d-95b7-de37458d63bf None None] MFA Rules not processed for user `cc99c93da5074cf5a48bc2aa9a92c5e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 22:13:58.046999 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-9264eaae-f812-4c0d-95b7-de37458d63bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:13:58.113247 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1023/2040] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jul 13 22:13:58 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 22:13:58.126752 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729bf3c4-14e2-45df-b1d6-d00d76cf8da5 admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:13:58.128895 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-729bf3c4-14e2-45df-b1d6-d00d76cf8da5 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:13:58.195094 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:13:58.199994 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-729bf3c4-14e2-45df-b1d6-d00d76cf8da5 admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:13:58.205699 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad77a4c-91a7-48d5-aea9-7dd6541954ac service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '847ec9a1bd4d4dec804318a5c845895a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417e3e2f1333481795a089bef2e24bea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:13:58.206413 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad77a4c-91a7-48d5-aea9-7dd6541954ac service neutron] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:13:58.206991 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad77a4c-91a7-48d5-aea9-7dd6541954ac service neutron] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:13:58.207173 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ad77a4c-91a7-48d5-aea9-7dd6541954ac service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:13:58.219800 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad77a4c-91a7-48d5-aea9-7dd6541954ac service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:13:58.221412 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ad77a4c-91a7-48d5-aea9-7dd6541954ac service neutron] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:13:58.233381 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1018/2041] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jul 13 22:13:58 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 22:13:58.797270 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9264eaae-f812-4c0d-95b7-de37458d63bf None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:13:58.798936 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-9264eaae-f812-4c0d-95b7-de37458d63bf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:13:58.812100 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9264eaae-f812-4c0d-95b7-de37458d63bf None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:13:58.817674 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ef015f6d-6f16-4bed-8bd5-28a46c48ac55 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:13:58.818905 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef015f6d-6f16-4bed-8bd5-28a46c48ac55 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:13:58.819179 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef015f6d-6f16-4bed-8bd5-28a46c48ac55 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:13:58.819452 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ef015f6d-6f16-4bed-8bd5-28a46c48ac55 admin admin] PATH_INFO: `/v3/projects/f95251554aaf49e39c01d1b367074fcf` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:13:58.820735 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef015f6d-6f16-4bed-8bd5-28a46c48ac55 admin admin] RBAC: Authorizing `identity:delete_project(project_id=f95251554aaf49e39c01d1b367074fcf)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:13:58.833773 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ef015f6d-6f16-4bed-8bd5-28a46c48ac55 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:13:58.849200 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=98,keystone:INSERT=2 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:13:58.905670 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-ef015f6d-6f16-4bed-8bd5-28a46c48ac55 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'f95251554aaf49e39c01d1b367074fcf', 'request_id': 'req-ef015f6d-6f16-4bed-8bd5-28a46c48ac55'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:13:58.913076 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-ef015f6d-6f16-4bed-8bd5-28a46c48ac55 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'f95251554aaf49e39c01d1b367074fcf', 'request_id': 'req-ef015f6d-6f16-4bed-8bd5-28a46c48ac55'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:13:58.920608 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-ef015f6d-6f16-4bed-8bd5-28a46c48ac55 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'f95251554aaf49e39c01d1b367074fcf', 'request_id': 'req-ef015f6d-6f16-4bed-8bd5-28a46c48ac55'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:13:58.940705 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-ef015f6d-6f16-4bed-8bd5-28a46c48ac55 admin admin] The token cache is being invalidate because project f95251554aaf49e39c01d1b367074fcf was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80381) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 22:13:58.941807 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-ef015f6d-6f16-4bed-8bd5-28a46c48ac55 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:13:58.942219 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad77a4c-91a7-48d5-aea9-7dd6541954ac service neutron] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:13:58.942705 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-ef015f6d-6f16-4bed-8bd5-28a46c48ac55 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:13:58.944468 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1024/2042] 10.222.0.52 () {58 vars in 1232 bytes} [Wed Jul 13 22:13:58 2022] DELETE /identity/v3/projects/f95251554aaf49e39c01d1b367074fcf => generated 0 bytes in 148 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 22:13:58.945260 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-4ad77a4c-91a7-48d5-aea9-7dd6541954ac service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:13:58.961692 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ad77a4c-91a7-48d5-aea9-7dd6541954ac service neutron] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:13:59.047390 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13139f4-06b1-4481-ad68-83c65ea82d90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:13:59.048345 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13139f4-06b1-4481-ad68-83c65ea82d90 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:13:59.048690 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13139f4-06b1-4481-ad68-83c65ea82d90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:13:59.049033 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c13139f4-06b1-4481-ad68-83c65ea82d90 admin admin] PATH_INFO: `/v3/users/ce9297274fe84e13a446c3b25e1a29bb` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:13:59.050458 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13139f4-06b1-4481-ad68-83c65ea82d90 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ce9297274fe84e13a446c3b25e1a29bb)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:13:59.053344 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c13139f4-06b1-4481-ad68-83c65ea82d90 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:13:59.124418 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-c13139f4-06b1-4481-ad68-83c65ea82d90 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ce9297274fe84e13a446c3b25e1a29bb'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:13:59.136678 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-c13139f4-06b1-4481-ad68-83c65ea82d90 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ce9297274fe84e13a446c3b25e1a29bb'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:13:59.155110 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-c13139f4-06b1-4481-ad68-83c65ea82d90 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ce9297274fe84e13a446c3b25e1a29bb'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:13:59.156198 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-c13139f4-06b1-4481-ad68-83c65ea82d90 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ce9297274fe84e13a446c3b25e1a29bb'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:13:59.156798 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-c13139f4-06b1-4481-ad68-83c65ea82d90 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ce9297274fe84e13a446c3b25e1a29bb'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:13:59.171640 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1019/2043] 10.222.0.52 () {58 vars in 1223 bytes} [Wed Jul 13 22:13:58 2022] DELETE /identity/v3/users/ce9297274fe84e13a446c3b25e1a29bb => generated 0 bytes in 231 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 13 22:13:59.176199 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b751d523-d3eb-4f30-b8df-c61747818f6d None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:13:59.176687 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b751d523-d3eb-4f30-b8df-c61747818f6d None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:13:59.176880 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b751d523-d3eb-4f30-b8df-c61747818f6d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:13:59.195613 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-b751d523-d3eb-4f30-b8df-c61747818f6d None None] MFA Rules not processed for user `cc99c93da5074cf5a48bc2aa9a92c5e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 22:13:59.196848 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-b751d523-d3eb-4f30-b8df-c61747818f6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:13:59.257327 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1025/2044] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jul 13 22:13:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 22:13:59.267143 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13139f4-06b1-4481-ad68-83c65ea82d90 admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:13:59.268609 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-c13139f4-06b1-4481-ad68-83c65ea82d90 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:13:59.328897 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c13139f4-06b1-4481-ad68-83c65ea82d90 admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:13:59.332946 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712f6340-d1ec-48fd-afe5-3fada640cd03 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '847ec9a1bd4d4dec804318a5c845895a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417e3e2f1333481795a089bef2e24bea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:13:59.333662 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712f6340-d1ec-48fd-afe5-3fada640cd03 service neutron] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:13:59.333930 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712f6340-d1ec-48fd-afe5-3fada640cd03 service neutron] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:13:59.334334 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-712f6340-d1ec-48fd-afe5-3fada640cd03 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:13:59.344141 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712f6340-d1ec-48fd-afe5-3fada640cd03 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:13:59.345754 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-712f6340-d1ec-48fd-afe5-3fada640cd03 service neutron] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:13:59.355692 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1020/2045] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jul 13 22:13:59 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 22:13:59.653978 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b751d523-d3eb-4f30-b8df-c61747818f6d None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:13:59.656161 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-b751d523-d3eb-4f30-b8df-c61747818f6d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:13:59.671473 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b751d523-d3eb-4f30-b8df-c61747818f6d None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:13:59.677661 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32f3615-5810-4446-a814-044ecc256ada admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:13:59.678923 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32f3615-5810-4446-a814-044ecc256ada admin admin] REQUEST_METHOD: `DELETE` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:13:59.679444 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32f3615-5810-4446-a814-044ecc256ada admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:13:59.680345 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c32f3615-5810-4446-a814-044ecc256ada admin admin] PATH_INFO: `/v3/projects/8ab45edd2d124f32817853049c50e467` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:13:59.681892 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32f3615-5810-4446-a814-044ecc256ada admin admin] RBAC: Authorizing `identity:delete_project(project_id=8ab45edd2d124f32817853049c50e467)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:13:59.684192 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c32f3615-5810-4446-a814-044ecc256ada admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:13:59.777143 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-c32f3615-5810-4446-a814-044ecc256ada admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8ab45edd2d124f32817853049c50e467', 'request_id': 'req-c32f3615-5810-4446-a814-044ecc256ada'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:13:59.788621 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-c32f3615-5810-4446-a814-044ecc256ada admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8ab45edd2d124f32817853049c50e467', 'request_id': 'req-c32f3615-5810-4446-a814-044ecc256ada'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:13:59.796540 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-c32f3615-5810-4446-a814-044ecc256ada admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8ab45edd2d124f32817853049c50e467', 'request_id': 'req-c32f3615-5810-4446-a814-044ecc256ada'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:13:59.831257 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-c32f3615-5810-4446-a814-044ecc256ada admin admin] The token cache is being invalidate because project 8ab45edd2d124f32817853049c50e467 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80381) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 22:13:59.831652 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-c32f3615-5810-4446-a814-044ecc256ada admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:13:59.832518 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-c32f3615-5810-4446-a814-044ecc256ada admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:13:59.835305 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1026/2046] 10.222.0.52 () {58 vars in 1232 bytes} [Wed Jul 13 22:13:59 2022] DELETE /identity/v3/projects/8ab45edd2d124f32817853049c50e467 => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 22:13:59.839818 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712f6340-d1ec-48fd-afe5-3fada640cd03 service neutron] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:13:59.854979 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-712f6340-d1ec-48fd-afe5-3fada640cd03 service neutron] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:13:59.865796 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:13:59.919093 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-90ee2238-bc03-46a8-ad0e-d9a8f0fd8a48 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:13:59.920000 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ee2238-bc03-46a8-ad0e-d9a8f0fd8a48 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:13:59.920300 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ee2238-bc03-46a8-ad0e-d9a8f0fd8a48 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:13:59.920568 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-90ee2238-bc03-46a8-ad0e-d9a8f0fd8a48 admin admin] PATH_INFO: `/v3/users/0de019a339ec4b0dae68f86c6358c6fe` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:13:59.921849 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ee2238-bc03-46a8-ad0e-d9a8f0fd8a48 admin admin] RBAC: Authorizing `identity:delete_user(user_id=0de019a339ec4b0dae68f86c6358c6fe)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:13:59.923823 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-90ee2238-bc03-46a8-ad0e-d9a8f0fd8a48 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:13:59.982642 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-90ee2238-bc03-46a8-ad0e-d9a8f0fd8a48 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '0de019a339ec4b0dae68f86c6358c6fe'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:13:59.996844 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-90ee2238-bc03-46a8-ad0e-d9a8f0fd8a48 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '0de019a339ec4b0dae68f86c6358c6fe'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:14:00.013471 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-90ee2238-bc03-46a8-ad0e-d9a8f0fd8a48 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '0de019a339ec4b0dae68f86c6358c6fe'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:14:00.014236 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-90ee2238-bc03-46a8-ad0e-d9a8f0fd8a48 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '0de019a339ec4b0dae68f86c6358c6fe'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:14:00.015013 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-90ee2238-bc03-46a8-ad0e-d9a8f0fd8a48 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '0de019a339ec4b0dae68f86c6358c6fe'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:14:00.029100 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1021/2047] 10.222.0.52 () {58 vars in 1223 bytes} [Wed Jul 13 22:13:59 2022] DELETE /identity/v3/users/0de019a339ec4b0dae68f86c6358c6fe => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 22:14:00.304320 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32f3615-5810-4446-a814-044ecc256ada admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:14:00.305938 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-c32f3615-5810-4446-a814-044ecc256ada admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:14:00.332370 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c32f3615-5810-4446-a814-044ecc256ada admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:14:00.351231 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:14:00.400729 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cd4ad378-b2ad-4367-ab79-5126b115bdc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:14:00.401655 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4ad378-b2ad-4367-ab79-5126b115bdc1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:14:00.401929 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4ad378-b2ad-4367-ab79-5126b115bdc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:14:00.402198 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cd4ad378-b2ad-4367-ab79-5126b115bdc1 admin admin] PATH_INFO: `/v3/projects/2a2b018a2a324f78b5f47ddbc73983fd` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:14:00.404082 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4ad378-b2ad-4367-ab79-5126b115bdc1 admin admin] RBAC: Authorizing `identity:delete_project(project_id=2a2b018a2a324f78b5f47ddbc73983fd)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:14:00.406040 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cd4ad378-b2ad-4367-ab79-5126b115bdc1 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:14:00.458704 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-cd4ad378-b2ad-4367-ab79-5126b115bdc1 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '2a2b018a2a324f78b5f47ddbc73983fd', 'request_id': 'req-cd4ad378-b2ad-4367-ab79-5126b115bdc1'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:14:00.468642 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-cd4ad378-b2ad-4367-ab79-5126b115bdc1 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '2a2b018a2a324f78b5f47ddbc73983fd', 'request_id': 'req-cd4ad378-b2ad-4367-ab79-5126b115bdc1'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:14:00.477417 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-cd4ad378-b2ad-4367-ab79-5126b115bdc1 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '2a2b018a2a324f78b5f47ddbc73983fd', 'request_id': 'req-cd4ad378-b2ad-4367-ab79-5126b115bdc1'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:14:00.492827 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-cd4ad378-b2ad-4367-ab79-5126b115bdc1 admin admin] The token cache is being invalidate because project 2a2b018a2a324f78b5f47ddbc73983fd was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80381) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 22:14:00.493117 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-cd4ad378-b2ad-4367-ab79-5126b115bdc1 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:14:00.493762 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-cd4ad378-b2ad-4367-ab79-5126b115bdc1 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:14:00.495401 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1027/2048] 10.222.0.52 () {58 vars in 1232 bytes} [Wed Jul 13 22:14:00 2022] DELETE /identity/v3/projects/2a2b018a2a324f78b5f47ddbc73983fd => generated 0 bytes in 192 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 22:14:00.500493 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a22179-86bb-4133-80ea-88140a544520 None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:14:00.500781 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a22179-86bb-4133-80ea-88140a544520 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:14:00.501065 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07a22179-86bb-4133-80ea-88140a544520 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:14:00.520154 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-07a22179-86bb-4133-80ea-88140a544520 None None] MFA Rules not processed for user `cc99c93da5074cf5a48bc2aa9a92c5e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 22:14:00.521379 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-07a22179-86bb-4133-80ea-88140a544520 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:14:00.586806 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1022/2049] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jul 13 22:14:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 22:14:00.594832 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4840e2-1332-4ff0-8737-72bcc8a0f0a5 None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:14:00.595124 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4840e2-1332-4ff0-8737-72bcc8a0f0a5 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:14:00.595421 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6b4840e2-1332-4ff0-8737-72bcc8a0f0a5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:14:00.629971 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-6b4840e2-1332-4ff0-8737-72bcc8a0f0a5 None None] MFA Rules not processed for user `cc99c93da5074cf5a48bc2aa9a92c5e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 22:14:00.631227 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-6b4840e2-1332-4ff0-8737-72bcc8a0f0a5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:14:00.639917 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1028/2050] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jul 13 22:14:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 46 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 22:14:00.644294 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a22179-86bb-4133-80ea-88140a544520 None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:14:00.657395 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07a22179-86bb-4133-80ea-88140a544520 None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:14:00.664364 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9d305e-1025-4162-82d9-84e4df823bb3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:14:00.665405 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9d305e-1025-4162-82d9-84e4df823bb3 admin admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:14:00.665767 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9d305e-1025-4162-82d9-84e4df823bb3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:14:00.666113 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d9d305e-1025-4162-82d9-84e4df823bb3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:14:00.667672 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d9d305e-1025-4162-82d9-84e4df823bb3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 22:14:00.668084 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d9d305e-1025-4162-82d9-84e4df823bb3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:14:00.669984 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d9d305e-1025-4162-82d9-84e4df823bb3 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:14:00.684840 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1023/2051] 10.222.0.52 () {58 vars in 1152 bytes} [Wed Jul 13 22:14:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 41 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 22:14:00.689730 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4840e2-1332-4ff0-8737-72bcc8a0f0a5 None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:14:00.704253 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6b4840e2-1332-4ff0-8737-72bcc8a0f0a5 None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:14:00.710443 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbbcaa5c-85a3-4eee-a98f-ded5190a9e2c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:14:00.711208 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbcaa5c-85a3-4eee-a98f-ded5190a9e2c admin admin] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:14:00.711489 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbcaa5c-85a3-4eee-a98f-ded5190a9e2c admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:14:00.711763 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbbcaa5c-85a3-4eee-a98f-ded5190a9e2c admin admin] PATH_INFO: `/v3/projects` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:14:00.712469 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbbcaa5c-85a3-4eee-a98f-ded5190a9e2c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:14:00.714384 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbbcaa5c-85a3-4eee-a98f-ded5190a9e2c admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:14:00.746340 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1029/2052] 10.222.0.52 () {60 vars in 1152 bytes} [Wed Jul 13 22:14:00 2022] POST /identity/v3/projects => generated 373 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 22:14:00.750806 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9d305e-1025-4162-82d9-84e4df823bb3 admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:14:00.764358 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d9d305e-1025-4162-82d9-84e4df823bb3 admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:14:00.783337 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:14:00.828549 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93647b3f-7c48-4979-a478-4857af954da8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:14:00.830351 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93647b3f-7c48-4979-a478-4857af954da8 admin admin] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:14:00.830351 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93647b3f-7c48-4979-a478-4857af954da8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:14:00.830351 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-93647b3f-7c48-4979-a478-4857af954da8 admin admin] PATH_INFO: `/v3/users` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:14:00.831063 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93647b3f-7c48-4979-a478-4857af954da8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:14:00.833626 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-93647b3f-7c48-4979-a478-4857af954da8 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:14:00.864204 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1024/2053] 10.222.0.52 () {60 vars in 1143 bytes} [Wed Jul 13 22:14:00 2022] POST /identity/v3/users => generated 287 bytes in 114 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 22:14:00.868409 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbbcaa5c-85a3-4eee-a98f-ded5190a9e2c admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:14:00.881898 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbbcaa5c-85a3-4eee-a98f-ded5190a9e2c admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:14:00.892085 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b592b288-cba1-4daf-bb7f-3e68845f43dc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:14:00.892803 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b592b288-cba1-4daf-bb7f-3e68845f43dc admin admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:14:00.893081 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b592b288-cba1-4daf-bb7f-3e68845f43dc admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:14:00.893359 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b592b288-cba1-4daf-bb7f-3e68845f43dc admin admin] PATH_INFO: `/v3/roles` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:14:00.894188 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b592b288-cba1-4daf-bb7f-3e68845f43dc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:14:00.896215 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b592b288-cba1-4daf-bb7f-3e68845f43dc admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:14:00.910635 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1030/2054] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jul 13 22:14:00 2022] GET /identity/v3/roles => generated 1420 bytes in 40 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 22:14:00.912031 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93647b3f-7c48-4979-a478-4857af954da8 admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:14:00.927035 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-93647b3f-7c48-4979-a478-4857af954da8 admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:14:00.932129 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d80ec992-0189-4c30-8819-626072f6104a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:14:00.933133 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80ec992-0189-4c30-8819-626072f6104a admin admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:14:00.933515 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80ec992-0189-4c30-8819-626072f6104a admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:14:00.934204 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d80ec992-0189-4c30-8819-626072f6104a admin admin] PATH_INFO: `/v3/roles` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:14:00.934908 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80ec992-0189-4c30-8819-626072f6104a admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:14:00.938712 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d80ec992-0189-4c30-8819-626072f6104a admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:14:00.955259 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1025/2055] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jul 13 22:14:00 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 22:14:00.959644 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b592b288-cba1-4daf-bb7f-3e68845f43dc admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:14:00.970428 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b592b288-cba1-4daf-bb7f-3e68845f43dc admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:14:00.978020 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b645a95a-528f-4a5d-b423-5b553de2e2a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:14:00.979012 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b645a95a-528f-4a5d-b423-5b553de2e2a6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:14:00.979294 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b645a95a-528f-4a5d-b423-5b553de2e2a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:14:00.979703 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b645a95a-528f-4a5d-b423-5b553de2e2a6 admin admin] PATH_INFO: `/v3/projects/30cbd56434a1422093cc5a8b39e2ad18/users/512f7b01a33c476aa3eb843fd369e983/roles/2a5e86976394498abb6310697cdb3f8b` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:14:01.000506 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b645a95a-528f-4a5d-b423-5b553de2e2a6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=30cbd56434a1422093cc5a8b39e2ad18, user_id=512f7b01a33c476aa3eb843fd369e983, role_id=2a5e86976394498abb6310697cdb3f8b)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:14:01.003484 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b645a95a-528f-4a5d-b423-5b553de2e2a6 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:14:01.024325 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1031/2056] 10.222.0.52 () {60 vars in 1482 bytes} [Wed Jul 13 22:14:00 2022] PUT /identity/v3/projects/30cbd56434a1422093cc5a8b39e2ad18/users/512f7b01a33c476aa3eb843fd369e983/roles/2a5e86976394498abb6310697cdb3f8b => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 22:14:01.034644 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f676d0-5281-4c80-b42c-375d6b535b65 None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:14:01.035149 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f676d0-5281-4c80-b42c-375d6b535b65 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:14:01.035341 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-77f676d0-5281-4c80-b42c-375d6b535b65 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:14:01.065347 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-77f676d0-5281-4c80-b42c-375d6b535b65 None None] MFA Rules not processed for user `512f7b01a33c476aa3eb843fd369e983`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 22:14:01.066695 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-77f676d0-5281-4c80-b42c-375d6b535b65 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:14:01.147688 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1026/2057] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jul 13 22:14:01 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 114 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 13 22:14:01.162803 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b645a95a-528f-4a5d-b423-5b553de2e2a6 admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:14:01.165681 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-b645a95a-528f-4a5d-b423-5b553de2e2a6 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:14:01.228008 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:14:01.234668 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b645a95a-528f-4a5d-b423-5b553de2e2a6 admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:14:01.243184 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bcf91866-4e56-44cc-8f0d-81d34559628c service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc77e1ea1ada4700b809beb1c980c5e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417e3e2f1333481795a089bef2e24bea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:14:01.243869 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf91866-4e56-44cc-8f0d-81d34559628c service cinder] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:14:01.244153 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf91866-4e56-44cc-8f0d-81d34559628c service cinder] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:14:01.244485 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bcf91866-4e56-44cc-8f0d-81d34559628c service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:14:01.255446 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf91866-4e56-44cc-8f0d-81d34559628c service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:14:01.257038 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bcf91866-4e56-44cc-8f0d-81d34559628c service cinder] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:14:01.267776 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1032/2058] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jul 13 22:14:01 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 22:14:01.288798 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f676d0-5281-4c80-b42c-375d6b535b65 None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:14:01.415640 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-77f676d0-5281-4c80-b42c-375d6b535b65 None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:14:01.420420 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab1eab1-c8fb-427e-b919-c59da9ac77b9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:14:01.421320 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab1eab1-c8fb-427e-b919-c59da9ac77b9 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:14:01.421605 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab1eab1-c8fb-427e-b919-c59da9ac77b9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:14:01.422061 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dab1eab1-c8fb-427e-b919-c59da9ac77b9 admin admin] PATH_INFO: `/v3/users/512f7b01a33c476aa3eb843fd369e983` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:14:01.423344 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab1eab1-c8fb-427e-b919-c59da9ac77b9 admin admin] RBAC: Authorizing `identity:delete_user(user_id=512f7b01a33c476aa3eb843fd369e983)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:14:01.425344 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dab1eab1-c8fb-427e-b919-c59da9ac77b9 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:14:01.491564 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=92,keystone:INSERT=3,keystone:DELETE=5 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:14:01.515408 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-dab1eab1-c8fb-427e-b919-c59da9ac77b9 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '512f7b01a33c476aa3eb843fd369e983'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:14:01.525495 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-dab1eab1-c8fb-427e-b919-c59da9ac77b9 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '512f7b01a33c476aa3eb843fd369e983'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:14:01.542156 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-dab1eab1-c8fb-427e-b919-c59da9ac77b9 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '512f7b01a33c476aa3eb843fd369e983'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:14:01.542995 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-dab1eab1-c8fb-427e-b919-c59da9ac77b9 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '512f7b01a33c476aa3eb843fd369e983'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:14:01.543733 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-dab1eab1-c8fb-427e-b919-c59da9ac77b9 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '512f7b01a33c476aa3eb843fd369e983'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:14:01.560670 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1027/2059] 10.222.0.52 () {58 vars in 1223 bytes} [Wed Jul 13 22:14:01 2022] DELETE /identity/v3/users/512f7b01a33c476aa3eb843fd369e983 => generated 0 bytes in 272 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 22:14:01.565264 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42c9ae1-4b99-4738-bfe3-0e62d271df11 None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:14:01.565847 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42c9ae1-4b99-4738-bfe3-0e62d271df11 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:14:01.565986 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f42c9ae1-4b99-4738-bfe3-0e62d271df11 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:14:01.589390 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-f42c9ae1-4b99-4738-bfe3-0e62d271df11 None None] MFA Rules not processed for user `cc99c93da5074cf5a48bc2aa9a92c5e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 22:14:01.591239 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-f42c9ae1-4b99-4738-bfe3-0e62d271df11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:14:01.652774 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1033/2060] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jul 13 22:14:01 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 22:14:01.666812 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab1eab1-c8fb-427e-b919-c59da9ac77b9 admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:14:01.668326 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-dab1eab1-c8fb-427e-b919-c59da9ac77b9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:14:01.734519 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dab1eab1-c8fb-427e-b919-c59da9ac77b9 admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:14:01.738573 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dfc5612-519a-4b1e-ae5e-5991d997f02f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '847ec9a1bd4d4dec804318a5c845895a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417e3e2f1333481795a089bef2e24bea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:14:01.739237 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfc5612-519a-4b1e-ae5e-5991d997f02f service neutron] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:14:01.739525 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfc5612-519a-4b1e-ae5e-5991d997f02f service neutron] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:14:01.739802 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfc5612-519a-4b1e-ae5e-5991d997f02f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:14:01.749660 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dfc5612-519a-4b1e-ae5e-5991d997f02f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:14:01.751206 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dfc5612-519a-4b1e-ae5e-5991d997f02f service neutron] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:14:01.762284 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1028/2061] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jul 13 22:14:01 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 22:14:02.793489 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42c9ae1-4b99-4738-bfe3-0e62d271df11 None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:14:02.795035 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-f42c9ae1-4b99-4738-bfe3-0e62d271df11 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:14:02.817583 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f42c9ae1-4b99-4738-bfe3-0e62d271df11 None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:14:02.824469 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ae5545bd-9e67-4dc9-8a6e-9ca026aae68d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:14:02.825463 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5545bd-9e67-4dc9-8a6e-9ca026aae68d admin admin] REQUEST_METHOD: `DELETE` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:14:02.825813 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5545bd-9e67-4dc9-8a6e-9ca026aae68d admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:14:02.826156 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ae5545bd-9e67-4dc9-8a6e-9ca026aae68d admin admin] PATH_INFO: `/v3/projects/30cbd56434a1422093cc5a8b39e2ad18` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:14:02.827787 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5545bd-9e67-4dc9-8a6e-9ca026aae68d admin admin] RBAC: Authorizing `identity:delete_project(project_id=30cbd56434a1422093cc5a8b39e2ad18)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:14:02.830441 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ae5545bd-9e67-4dc9-8a6e-9ca026aae68d admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:14:02.880854 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-ae5545bd-9e67-4dc9-8a6e-9ca026aae68d admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '30cbd56434a1422093cc5a8b39e2ad18', 'request_id': 'req-ae5545bd-9e67-4dc9-8a6e-9ca026aae68d'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:14:02.887911 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-ae5545bd-9e67-4dc9-8a6e-9ca026aae68d admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '30cbd56434a1422093cc5a8b39e2ad18', 'request_id': 'req-ae5545bd-9e67-4dc9-8a6e-9ca026aae68d'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:14:02.894116 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-ae5545bd-9e67-4dc9-8a6e-9ca026aae68d admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '30cbd56434a1422093cc5a8b39e2ad18', 'request_id': 'req-ae5545bd-9e67-4dc9-8a6e-9ca026aae68d'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:14:02.909013 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-ae5545bd-9e67-4dc9-8a6e-9ca026aae68d admin admin] The token cache is being invalidate because project 30cbd56434a1422093cc5a8b39e2ad18 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80381) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 22:14:02.909525 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-ae5545bd-9e67-4dc9-8a6e-9ca026aae68d admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:14:02.910817 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-ae5545bd-9e67-4dc9-8a6e-9ca026aae68d admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:14:02.912521 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1034/2062] 10.222.0.52 () {58 vars in 1232 bytes} [Wed Jul 13 22:14:02 2022] DELETE /identity/v3/projects/30cbd56434a1422093cc5a8b39e2ad18 => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 22:14:02.918559 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa653fd5-9222-48a7-8148-9b23be1445ae None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:14:02.919100 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa653fd5-9222-48a7-8148-9b23be1445ae None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:14:02.919100 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fa653fd5-9222-48a7-8148-9b23be1445ae None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:14:02.938183 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-fa653fd5-9222-48a7-8148-9b23be1445ae None None] MFA Rules not processed for user `cc99c93da5074cf5a48bc2aa9a92c5e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 22:14:02.939525 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-fa653fd5-9222-48a7-8148-9b23be1445ae None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:14:03.007659 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1029/2063] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jul 13 22:14:02 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 22:14:03.016456 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4500839f-818f-4e4c-8087-fb3365615041 None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:14:03.016746 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4500839f-818f-4e4c-8087-fb3365615041 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:14:03.017111 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4500839f-818f-4e4c-8087-fb3365615041 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:14:03.036172 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-4500839f-818f-4e4c-8087-fb3365615041 None None] MFA Rules not processed for user `cc99c93da5074cf5a48bc2aa9a92c5e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 22:14:03.037415 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-4500839f-818f-4e4c-8087-fb3365615041 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:14:03.046052 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1035/2064] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jul 13 22:14:03 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 22:14:03.050770 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa653fd5-9222-48a7-8148-9b23be1445ae None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:14:03.060598 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fa653fd5-9222-48a7-8148-9b23be1445ae None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:14:03.065675 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5043d2be-25e4-435f-93bb-8b02f693e6ea admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:14:03.066561 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5043d2be-25e4-435f-93bb-8b02f693e6ea admin admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:14:03.066884 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5043d2be-25e4-435f-93bb-8b02f693e6ea admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:14:03.067160 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5043d2be-25e4-435f-93bb-8b02f693e6ea admin admin] PATH_INFO: `/v3/domains` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:14:03.067935 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5043d2be-25e4-435f-93bb-8b02f693e6ea admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 22:14:03.068243 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5043d2be-25e4-435f-93bb-8b02f693e6ea admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:14:03.069951 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5043d2be-25e4-435f-93bb-8b02f693e6ea admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:14:03.084739 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1030/2065] 10.222.0.52 () {58 vars in 1152 bytes} [Wed Jul 13 22:14:03 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 22:14:03.089011 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4500839f-818f-4e4c-8087-fb3365615041 None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:14:03.099180 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4500839f-818f-4e4c-8087-fb3365615041 None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:14:03.104601 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110954df-fa29-4f64-ba9a-6efef24a4fae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:14:03.105276 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110954df-fa29-4f64-ba9a-6efef24a4fae admin admin] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:14:03.105556 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110954df-fa29-4f64-ba9a-6efef24a4fae admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:14:03.105931 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-110954df-fa29-4f64-ba9a-6efef24a4fae admin admin] PATH_INFO: `/v3/projects` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:14:03.106666 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110954df-fa29-4f64-ba9a-6efef24a4fae admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:14:03.108606 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-110954df-fa29-4f64-ba9a-6efef24a4fae admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:14:03.147338 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1036/2066] 10.222.0.52 () {60 vars in 1152 bytes} [Wed Jul 13 22:14:03 2022] POST /identity/v3/projects => generated 365 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 22:14:03.151746 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5043d2be-25e4-435f-93bb-8b02f693e6ea admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:14:03.162183 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5043d2be-25e4-435f-93bb-8b02f693e6ea admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:14:03.196971 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=94,keystone:DELETE=5,keystone:INSERT=1 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:14:03.223222 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb123b79-fcf4-41e1-afa2-01330d1db2d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:14:03.223901 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb123b79-fcf4-41e1-afa2-01330d1db2d0 admin admin] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:14:03.224339 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb123b79-fcf4-41e1-afa2-01330d1db2d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:14:03.224623 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cb123b79-fcf4-41e1-afa2-01330d1db2d0 admin admin] PATH_INFO: `/v3/users` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:14:03.225324 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb123b79-fcf4-41e1-afa2-01330d1db2d0 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:14:03.227314 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cb123b79-fcf4-41e1-afa2-01330d1db2d0 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:14:03.260619 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1031/2067] 10.222.0.52 () {60 vars in 1143 bytes} [Wed Jul 13 22:14:03 2022] POST /identity/v3/users => generated 283 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 22:14:03.262308 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110954df-fa29-4f64-ba9a-6efef24a4fae admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:14:03.273832 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-110954df-fa29-4f64-ba9a-6efef24a4fae admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:14:03.280043 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e2faa5-7751-4d30-9302-37dfe8ff6521 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:14:03.280718 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e2faa5-7751-4d30-9302-37dfe8ff6521 admin admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:14:03.280995 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e2faa5-7751-4d30-9302-37dfe8ff6521 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:14:03.281272 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-27e2faa5-7751-4d30-9302-37dfe8ff6521 admin admin] PATH_INFO: `/v3/roles` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:14:03.281986 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e2faa5-7751-4d30-9302-37dfe8ff6521 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:14:03.284151 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-27e2faa5-7751-4d30-9302-37dfe8ff6521 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:14:03.298639 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1037/2068] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jul 13 22:14:03 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 22:14:03.299784 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb123b79-fcf4-41e1-afa2-01330d1db2d0 admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:14:03.310909 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cb123b79-fcf4-41e1-afa2-01330d1db2d0 admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:14:03.315618 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7e3ba8a-8342-4908-8532-14b7d3042ac3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:14:03.316363 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e3ba8a-8342-4908-8532-14b7d3042ac3 admin admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:14:03.316643 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e3ba8a-8342-4908-8532-14b7d3042ac3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:14:03.316915 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7e3ba8a-8342-4908-8532-14b7d3042ac3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:14:03.317645 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e3ba8a-8342-4908-8532-14b7d3042ac3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:14:03.319546 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7e3ba8a-8342-4908-8532-14b7d3042ac3 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:14:03.337082 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1032/2069] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jul 13 22:14:03 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 22:14:03.341479 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e2faa5-7751-4d30-9302-37dfe8ff6521 admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:14:03.360510 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-27e2faa5-7751-4d30-9302-37dfe8ff6521 admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:14:03.365277 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5e3a6a-a828-450b-8126-2e94fd3ea208 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:14:03.366223 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5e3a6a-a828-450b-8126-2e94fd3ea208 admin admin] REQUEST_METHOD: `PUT` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:14:03.366513 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5e3a6a-a828-450b-8126-2e94fd3ea208 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:14:03.366815 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe5e3a6a-a828-450b-8126-2e94fd3ea208 admin admin] PATH_INFO: `/v3/projects/3d82e209d4504d6d81e3b54444fb59ed/users/96b12071c9bd45baba003ec6253136a1/roles/2a5e86976394498abb6310697cdb3f8b` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:14:03.395936 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5e3a6a-a828-450b-8126-2e94fd3ea208 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3d82e209d4504d6d81e3b54444fb59ed, user_id=96b12071c9bd45baba003ec6253136a1, role_id=2a5e86976394498abb6310697cdb3f8b)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:14:03.399255 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe5e3a6a-a828-450b-8126-2e94fd3ea208 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:14:03.418798 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1038/2070] 10.222.0.52 () {60 vars in 1482 bytes} [Wed Jul 13 22:14:03 2022] PUT /identity/v3/projects/3d82e209d4504d6d81e3b54444fb59ed/users/96b12071c9bd45baba003ec6253136a1/roles/2a5e86976394498abb6310697cdb3f8b => generated 0 bytes in 76 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 13 22:14:03.430953 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31641e20-3120-40c0-9746-7ff6b6f1af69 None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:14:03.431359 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31641e20-3120-40c0-9746-7ff6b6f1af69 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:14:03.431757 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-31641e20-3120-40c0-9746-7ff6b6f1af69 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:14:03.466870 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-31641e20-3120-40c0-9746-7ff6b6f1af69 None None] MFA Rules not processed for user `96b12071c9bd45baba003ec6253136a1`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 22:14:03.468282 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-31641e20-3120-40c0-9746-7ff6b6f1af69 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:14:03.565156 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1033/2071] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jul 13 22:14:03 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 135 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 22:14:03.573697 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5e3a6a-a828-450b-8126-2e94fd3ea208 admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:14:03.575623 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-fe5e3a6a-a828-450b-8126-2e94fd3ea208 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:14:03.588975 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:14:03.643358 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe5e3a6a-a828-450b-8126-2e94fd3ea208 admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:14:03.647358 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-11fe95fb-88c6-4a19-a8b1-80bcf5a76135 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc77e1ea1ada4700b809beb1c980c5e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417e3e2f1333481795a089bef2e24bea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:14:03.648013 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fe95fb-88c6-4a19-a8b1-80bcf5a76135 service cinder] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:14:03.648288 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fe95fb-88c6-4a19-a8b1-80bcf5a76135 service cinder] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:14:03.648558 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-11fe95fb-88c6-4a19-a8b1-80bcf5a76135 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:14:03.659392 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11fe95fb-88c6-4a19-a8b1-80bcf5a76135 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:14:03.661142 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-11fe95fb-88c6-4a19-a8b1-80bcf5a76135 service cinder] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:14:03.671417 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1039/2072] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jul 13 22:14:03 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 22:14:10.400110 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd2378d-4fd3-4152-a173-73f1605806d6 None None] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:14:10.400629 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd2378d-4fd3-4152-a173-73f1605806d6 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:14:10.400891 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bd2378d-4fd3-4152-a173-73f1605806d6 None None] PATH_INFO: `/` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:14:10.402392 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1034/2073] 10.222.0.52 () {54 vars in 921 bytes} [Wed Jul 13 22:14:10 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 22:14:10.437003 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516660be-4404-459a-86c2-b634e4db536d None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:14:10.437501 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516660be-4404-459a-86c2-b634e4db536d None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:14:10.437661 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-516660be-4404-459a-86c2-b634e4db536d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:14:10.461998 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-516660be-4404-459a-86c2-b634e4db536d None None] MFA Rules not processed for user `ac673d641d8849c79bf4cf71c9805587`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 22:14:10.463493 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-516660be-4404-459a-86c2-b634e4db536d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:14:10.478243 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd2378d-4fd3-4152-a173-73f1605806d6 None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:14:10.480049 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-7bd2378d-4fd3-4152-a173-73f1605806d6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:14:10.557361 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1040/2074] 10.222.0.52 () {58 vars in 1018 bytes} [Wed Jul 13 22:14:10 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 121 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 22:14:10.570773 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516660be-4404-459a-86c2-b634e4db536d None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:14:10.572999 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-516660be-4404-459a-86c2-b634e4db536d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:14:10.575663 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bd2378d-4fd3-4152-a173-73f1605806d6 None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:14:10.581857 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d020db-f40a-4b1f-b9d1-1a87fcf976f1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:14:10.583434 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d020db-f40a-4b1f-b9d1-1a87fcf976f1 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:14:10.583798 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d020db-f40a-4b1f-b9d1-1a87fcf976f1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:14:10.584086 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d020db-f40a-4b1f-b9d1-1a87fcf976f1 admin admin] PATH_INFO: `/v3/users/99a16a550cdf407880fde2ba580fa0c2` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:14:10.585585 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d020db-f40a-4b1f-b9d1-1a87fcf976f1 admin admin] RBAC: Authorizing `identity:delete_user(user_id=99a16a550cdf407880fde2ba580fa0c2)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:14:10.587644 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-516660be-4404-459a-86c2-b634e4db536d None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:14:10.588115 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d020db-f40a-4b1f-b9d1-1a87fcf976f1 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:14:10.593062 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003a7dff-3216-42ea-9968-ddf40fa849ed service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac673d641d8849c79bf4cf71c9805587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417e3e2f1333481795a089bef2e24bea', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:14:10.593727 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003a7dff-3216-42ea-9968-ddf40fa849ed service placement] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:14:10.594016 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003a7dff-3216-42ea-9968-ddf40fa849ed service placement] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:14:10.594284 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-003a7dff-3216-42ea-9968-ddf40fa849ed service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:14:10.604289 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-003a7dff-3216-42ea-9968-ddf40fa849ed service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:14:10.605934 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-003a7dff-3216-42ea-9968-ddf40fa849ed service placement] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:14:10.618080 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1041/2075] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jul 13 22:14:10 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 48 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 13 22:14:10.642887 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-53d020db-f40a-4b1f-b9d1-1a87fcf976f1 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '99a16a550cdf407880fde2ba580fa0c2'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:14:10.646515 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=90,keystone:INSERT=3,keystone:DELETE=7 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:14:10.660049 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-53d020db-f40a-4b1f-b9d1-1a87fcf976f1 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '99a16a550cdf407880fde2ba580fa0c2'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:14:10.679606 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-53d020db-f40a-4b1f-b9d1-1a87fcf976f1 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '99a16a550cdf407880fde2ba580fa0c2'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:14:10.680427 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-53d020db-f40a-4b1f-b9d1-1a87fcf976f1 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '99a16a550cdf407880fde2ba580fa0c2'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:14:10.681139 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-53d020db-f40a-4b1f-b9d1-1a87fcf976f1 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '99a16a550cdf407880fde2ba580fa0c2'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:14:10.696198 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1035/2076] 10.222.0.52 () {58 vars in 1223 bytes} [Wed Jul 13 22:14:10 2022] DELETE /identity/v3/users/99a16a550cdf407880fde2ba580fa0c2 => generated 0 bytes in 218 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 13 22:14:10.900590 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003a7dff-3216-42ea-9968-ddf40fa849ed service placement] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:14:10.912148 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-003a7dff-3216-42ea-9968-ddf40fa849ed service placement] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:14:10.975905 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c4b53b8f-85e6-48c5-a731-2e66a91b5dca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:14:10.976782 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b53b8f-85e6-48c5-a731-2e66a91b5dca admin admin] REQUEST_METHOD: `DELETE` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:14:10.977063 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b53b8f-85e6-48c5-a731-2e66a91b5dca admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:14:10.977336 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c4b53b8f-85e6-48c5-a731-2e66a91b5dca admin admin] PATH_INFO: `/v3/projects/3035484baa324b01b7aa94b3322d9e6e` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:14:10.978552 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b53b8f-85e6-48c5-a731-2e66a91b5dca admin admin] RBAC: Authorizing `identity:delete_project(project_id=3035484baa324b01b7aa94b3322d9e6e)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:14:10.980533 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c4b53b8f-85e6-48c5-a731-2e66a91b5dca admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:14:11.020337 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:14:11.030284 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-c4b53b8f-85e6-48c5-a731-2e66a91b5dca admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3035484baa324b01b7aa94b3322d9e6e', 'request_id': 'req-c4b53b8f-85e6-48c5-a731-2e66a91b5dca'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:14:11.036366 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-c4b53b8f-85e6-48c5-a731-2e66a91b5dca admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3035484baa324b01b7aa94b3322d9e6e', 'request_id': 'req-c4b53b8f-85e6-48c5-a731-2e66a91b5dca'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:14:11.042863 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-c4b53b8f-85e6-48c5-a731-2e66a91b5dca admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3035484baa324b01b7aa94b3322d9e6e', 'request_id': 'req-c4b53b8f-85e6-48c5-a731-2e66a91b5dca'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:14:11.057785 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-c4b53b8f-85e6-48c5-a731-2e66a91b5dca admin admin] The token cache is being invalidate because project 3035484baa324b01b7aa94b3322d9e6e was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80381) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 22:14:11.058053 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-c4b53b8f-85e6-48c5-a731-2e66a91b5dca admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:14:11.058712 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-c4b53b8f-85e6-48c5-a731-2e66a91b5dca admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:14:11.060272 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1042/2077] 10.222.0.52 () {58 vars in 1232 bytes} [Wed Jul 13 22:14:10 2022] DELETE /identity/v3/projects/3035484baa324b01b7aa94b3322d9e6e => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 22:14:11.068209 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731c040b-cc24-42f6-ac9c-4a9b822b7371 None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:14:11.068587 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731c040b-cc24-42f6-ac9c-4a9b822b7371 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:14:11.068915 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-731c040b-cc24-42f6-ac9c-4a9b822b7371 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:14:11.088216 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-731c040b-cc24-42f6-ac9c-4a9b822b7371 None None] MFA Rules not processed for user `cc99c93da5074cf5a48bc2aa9a92c5e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 22:14:11.089442 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-731c040b-cc24-42f6-ac9c-4a9b822b7371 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:14:11.152893 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1036/2078] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jul 13 22:14:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 22:14:11.163776 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339bc7bd-0e4c-4ca7-b973-b75edcb46f0d None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:14:11.164067 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339bc7bd-0e4c-4ca7-b973-b75edcb46f0d None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:14:11.164343 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-339bc7bd-0e4c-4ca7-b973-b75edcb46f0d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:14:11.184441 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-339bc7bd-0e4c-4ca7-b973-b75edcb46f0d None None] MFA Rules not processed for user `cc99c93da5074cf5a48bc2aa9a92c5e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 22:14:11.185656 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-339bc7bd-0e4c-4ca7-b973-b75edcb46f0d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:14:11.195121 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1043/2079] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jul 13 22:14:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 22:14:11.201696 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731c040b-cc24-42f6-ac9c-4a9b822b7371 None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:14:11.212897 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-731c040b-cc24-42f6-ac9c-4a9b822b7371 None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:14:11.218791 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4290276b-081f-45cb-8ed4-64263a55f645 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:14:11.219463 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4290276b-081f-45cb-8ed4-64263a55f645 admin admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:14:11.219743 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4290276b-081f-45cb-8ed4-64263a55f645 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:14:11.220018 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4290276b-081f-45cb-8ed4-64263a55f645 admin admin] PATH_INFO: `/v3/domains` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:14:11.220764 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4290276b-081f-45cb-8ed4-64263a55f645 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 22:14:11.221069 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4290276b-081f-45cb-8ed4-64263a55f645 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:14:11.222750 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4290276b-081f-45cb-8ed4-64263a55f645 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:14:11.235465 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1037/2080] 10.222.0.52 () {58 vars in 1152 bytes} [Wed Jul 13 22:14:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 22:14:11.240010 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339bc7bd-0e4c-4ca7-b973-b75edcb46f0d None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:14:11.249833 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-339bc7bd-0e4c-4ca7-b973-b75edcb46f0d None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:14:11.255526 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250a2117-7330-4083-9c65-f9aa2a2314a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:14:11.256110 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250a2117-7330-4083-9c65-f9aa2a2314a3 admin admin] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:14:11.256354 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250a2117-7330-4083-9c65-f9aa2a2314a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:14:11.256660 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-250a2117-7330-4083-9c65-f9aa2a2314a3 admin admin] PATH_INFO: `/v3/projects` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:14:11.257346 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250a2117-7330-4083-9c65-f9aa2a2314a3 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:14:11.259356 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-250a2117-7330-4083-9c65-f9aa2a2314a3 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:14:11.287413 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1044/2081] 10.222.0.52 () {60 vars in 1152 bytes} [Wed Jul 13 22:14:11 2022] POST /identity/v3/projects => generated 371 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 22:14:11.291768 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4290276b-081f-45cb-8ed4-64263a55f645 admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:14:11.302662 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4290276b-081f-45cb-8ed4-64263a55f645 admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:14:11.363225 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f15027-7c62-4e27-af51-62cff5603916 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:14:11.363913 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f15027-7c62-4e27-af51-62cff5603916 admin admin] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:14:11.364246 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f15027-7c62-4e27-af51-62cff5603916 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:14:11.364519 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b6f15027-7c62-4e27-af51-62cff5603916 admin admin] PATH_INFO: `/v3/users` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:14:11.365239 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f15027-7c62-4e27-af51-62cff5603916 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:14:11.367205 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b6f15027-7c62-4e27-af51-62cff5603916 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:14:11.390178 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1038/2082] 10.222.0.52 () {60 vars in 1143 bytes} [Wed Jul 13 22:14:11 2022] POST /identity/v3/users => generated 286 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 13 22:14:11.394304 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250a2117-7330-4083-9c65-f9aa2a2314a3 admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:14:11.405326 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-250a2117-7330-4083-9c65-f9aa2a2314a3 admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:14:11.411094 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d28069e-c0cf-42ce-a2cc-af2b11693d10 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:14:11.411874 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d28069e-c0cf-42ce-a2cc-af2b11693d10 admin admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:14:11.412154 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d28069e-c0cf-42ce-a2cc-af2b11693d10 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:14:11.412427 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d28069e-c0cf-42ce-a2cc-af2b11693d10 admin admin] PATH_INFO: `/v3/roles` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:14:11.413159 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d28069e-c0cf-42ce-a2cc-af2b11693d10 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:14:11.414794 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d28069e-c0cf-42ce-a2cc-af2b11693d10 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:14:11.425446 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1045/2083] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jul 13 22:14:11 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 22:14:11.430249 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f15027-7c62-4e27-af51-62cff5603916 admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:14:11.441627 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b6f15027-7c62-4e27-af51-62cff5603916 admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:14:11.458795 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f8f655c-f1ba-465d-b2fc-ada9adf86dc1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:14:11.458795 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8f655c-f1ba-465d-b2fc-ada9adf86dc1 admin admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:14:11.458795 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8f655c-f1ba-465d-b2fc-ada9adf86dc1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:14:11.458795 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f8f655c-f1ba-465d-b2fc-ada9adf86dc1 admin admin] PATH_INFO: `/v3/roles` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:14:11.459158 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f8f655c-f1ba-465d-b2fc-ada9adf86dc1 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:14:11.459158 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f8f655c-f1ba-465d-b2fc-ada9adf86dc1 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:14:11.470907 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1039/2084] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jul 13 22:14:11 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 22:14:11.472115 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d28069e-c0cf-42ce-a2cc-af2b11693d10 admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:14:11.484130 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d28069e-c0cf-42ce-a2cc-af2b11693d10 admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:14:11.489036 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe08bc6-b14f-4431-ae4a-c197eb871770 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:14:11.490190 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe08bc6-b14f-4431-ae4a-c197eb871770 admin admin] REQUEST_METHOD: `PUT` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:14:11.490563 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe08bc6-b14f-4431-ae4a-c197eb871770 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:14:11.490902 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cbe08bc6-b14f-4431-ae4a-c197eb871770 admin admin] PATH_INFO: `/v3/projects/3c17de2a479e4c4995e71fef83345796/users/b49852268c87452bb8a35116c5185e94/roles/2a5e86976394498abb6310697cdb3f8b` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:14:11.506736 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe08bc6-b14f-4431-ae4a-c197eb871770 admin admin] RBAC: Authorizing `identity:create_grant(project_id=3c17de2a479e4c4995e71fef83345796, user_id=b49852268c87452bb8a35116c5185e94, role_id=2a5e86976394498abb6310697cdb3f8b)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:14:11.509527 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cbe08bc6-b14f-4431-ae4a-c197eb871770 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:14:11.526353 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1046/2085] 10.222.0.52 () {60 vars in 1482 bytes} [Wed Jul 13 22:14:11 2022] PUT /identity/v3/projects/3c17de2a479e4c4995e71fef83345796/users/b49852268c87452bb8a35116c5185e94/roles/2a5e86976394498abb6310697cdb3f8b => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 22:14:11.537175 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cb599e-c43b-4b8d-85b8-f37ae09da42c None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:14:11.537520 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cb599e-c43b-4b8d-85b8-f37ae09da42c None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:14:11.537735 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60cb599e-c43b-4b8d-85b8-f37ae09da42c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:14:11.570746 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-60cb599e-c43b-4b8d-85b8-f37ae09da42c None None] MFA Rules not processed for user `b49852268c87452bb8a35116c5185e94`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 22:14:11.572456 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-60cb599e-c43b-4b8d-85b8-f37ae09da42c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:14:11.593352 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:INSERT=4,keystone:DELETE=3,keystone:SELECT=93 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:14:11.645854 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1040/2086] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jul 13 22:14:11 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 22:14:11.651246 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe08bc6-b14f-4431-ae4a-c197eb871770 admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:14:11.716539 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cbe08bc6-b14f-4431-ae4a-c197eb871770 admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:14:11.721462 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771c0946-64d6-44cd-9c58-4c08bf320b13 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:14:11.722134 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771c0946-64d6-44cd-9c58-4c08bf320b13 admin admin] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:14:11.722415 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771c0946-64d6-44cd-9c58-4c08bf320b13 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:14:11.722723 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-771c0946-64d6-44cd-9c58-4c08bf320b13 admin admin] PATH_INFO: `/v3/projects` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:14:11.723505 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771c0946-64d6-44cd-9c58-4c08bf320b13 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:14:11.725438 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-771c0946-64d6-44cd-9c58-4c08bf320b13 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:14:11.753408 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1047/2087] 10.222.0.52 () {60 vars in 1152 bytes} [Wed Jul 13 22:14:11 2022] POST /identity/v3/projects => generated 371 bytes in 103 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 22:14:11.757591 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60cb599e-c43b-4b8d-85b8-f37ae09da42c None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:14:11.769876 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60cb599e-c43b-4b8d-85b8-f37ae09da42c None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:14:11.834988 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff53d3d-55ef-4f46-a1df-73175f56c207 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:14:11.835676 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff53d3d-55ef-4f46-a1df-73175f56c207 admin admin] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:14:11.836090 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff53d3d-55ef-4f46-a1df-73175f56c207 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:14:11.836486 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ff53d3d-55ef-4f46-a1df-73175f56c207 admin admin] PATH_INFO: `/v3/users` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:14:11.837205 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff53d3d-55ef-4f46-a1df-73175f56c207 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:14:11.839197 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ff53d3d-55ef-4f46-a1df-73175f56c207 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:14:11.860773 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1041/2088] 10.222.0.52 () {60 vars in 1143 bytes} [Wed Jul 13 22:14:11 2022] POST /identity/v3/users => generated 292 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 13 22:14:11.865523 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771c0946-64d6-44cd-9c58-4c08bf320b13 admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:14:11.876003 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-771c0946-64d6-44cd-9c58-4c08bf320b13 admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:14:11.880834 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b5155c-d8ee-483a-92ad-5083f5ce0156 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:14:11.881529 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b5155c-d8ee-483a-92ad-5083f5ce0156 admin admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:14:11.881799 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b5155c-d8ee-483a-92ad-5083f5ce0156 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:14:11.882083 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5b5155c-d8ee-483a-92ad-5083f5ce0156 admin admin] PATH_INFO: `/v3/roles` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:14:11.882821 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b5155c-d8ee-483a-92ad-5083f5ce0156 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:14:11.884426 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5b5155c-d8ee-483a-92ad-5083f5ce0156 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:14:11.897356 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1048/2089] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jul 13 22:14:11 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 22:14:11.901358 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff53d3d-55ef-4f46-a1df-73175f56c207 admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:14:11.912048 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ff53d3d-55ef-4f46-a1df-73175f56c207 admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:14:11.916812 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94c2880-e1b8-4bd8-9615-6fbb34987059 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:14:11.917702 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94c2880-e1b8-4bd8-9615-6fbb34987059 admin admin] REQUEST_METHOD: `PUT` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:14:11.917980 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94c2880-e1b8-4bd8-9615-6fbb34987059 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:14:11.918250 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c94c2880-e1b8-4bd8-9615-6fbb34987059 admin admin] PATH_INFO: `/v3/projects/a19b4e202eec402c9c6283afe2865a3a/users/fafca19dfa4640529c12740e7853639a/roles/44efba2579f342df8a0789c59d163ec3` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:14:11.935818 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94c2880-e1b8-4bd8-9615-6fbb34987059 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a19b4e202eec402c9c6283afe2865a3a, user_id=fafca19dfa4640529c12740e7853639a, role_id=44efba2579f342df8a0789c59d163ec3)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:14:11.938374 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c94c2880-e1b8-4bd8-9615-6fbb34987059 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:14:11.965627 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1042/2090] 10.222.0.52 () {60 vars in 1482 bytes} [Wed Jul 13 22:14:11 2022] PUT /identity/v3/projects/a19b4e202eec402c9c6283afe2865a3a/users/fafca19dfa4640529c12740e7853639a/roles/44efba2579f342df8a0789c59d163ec3 => generated 0 bytes in 65 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 22:14:11.971434 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b5155c-d8ee-483a-92ad-5083f5ce0156 admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:14:11.983884 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5b5155c-d8ee-483a-92ad-5083f5ce0156 admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:14:12.019914 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:14:12.046807 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-037d5266-904b-4738-a34a-a06a495c061c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:14:12.047764 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037d5266-904b-4738-a34a-a06a495c061c admin admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:14:12.048051 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037d5266-904b-4738-a34a-a06a495c061c admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:14:12.048362 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-037d5266-904b-4738-a34a-a06a495c061c admin admin] PATH_INFO: `/v3/roles` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:14:12.049107 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-037d5266-904b-4738-a34a-a06a495c061c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:14:12.050819 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-037d5266-904b-4738-a34a-a06a495c061c admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:14:12.061399 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1049/2091] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jul 13 22:14:11 2022] GET /identity/v3/roles => generated 1420 bytes in 91 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 22:14:12.065721 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94c2880-e1b8-4bd8-9615-6fbb34987059 admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:14:12.077062 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c94c2880-e1b8-4bd8-9615-6fbb34987059 admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:14:12.082051 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df932ed-2e37-4711-b9b3-792668716ec3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:14:12.083042 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df932ed-2e37-4711-b9b3-792668716ec3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:14:12.083395 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df932ed-2e37-4711-b9b3-792668716ec3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:14:12.083671 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df932ed-2e37-4711-b9b3-792668716ec3 admin admin] PATH_INFO: `/v3/projects/a19b4e202eec402c9c6283afe2865a3a/users/fafca19dfa4640529c12740e7853639a/roles/44efba2579f342df8a0789c59d163ec3` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:14:12.085736 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df932ed-2e37-4711-b9b3-792668716ec3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a19b4e202eec402c9c6283afe2865a3a, user_id=fafca19dfa4640529c12740e7853639a, role_id=44efba2579f342df8a0789c59d163ec3)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:14:12.088382 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df932ed-2e37-4711-b9b3-792668716ec3 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:14:12.100021 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1043/2092] 10.222.0.52 () {60 vars in 1482 bytes} [Wed Jul 13 22:14:12 2022] PUT /identity/v3/projects/a19b4e202eec402c9c6283afe2865a3a/users/fafca19dfa4640529c12740e7853639a/roles/44efba2579f342df8a0789c59d163ec3 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 13 22:14:12.111397 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce79401-1c8c-47bb-bf4f-99b9a68a90d4 None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:14:12.111690 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce79401-1c8c-47bb-bf4f-99b9a68a90d4 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:14:12.111994 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bce79401-1c8c-47bb-bf4f-99b9a68a90d4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:14:12.140594 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-bce79401-1c8c-47bb-bf4f-99b9a68a90d4 None None] MFA Rules not processed for user `fafca19dfa4640529c12740e7853639a`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 22:14:12.141832 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-bce79401-1c8c-47bb-bf4f-99b9a68a90d4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:14:12.212892 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1050/2093] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jul 13 22:14:12 2022] POST /identity/v3/auth/tokens => generated 3305 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 22:14:12.225908 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df932ed-2e37-4711-b9b3-792668716ec3 admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:14:12.228570 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-2df932ed-2e37-4711-b9b3-792668716ec3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:14:12.296946 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df932ed-2e37-4711-b9b3-792668716ec3 admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:14:12.313185 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9533060-e083-4875-8639-2707f9e222bc service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc77e1ea1ada4700b809beb1c980c5e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417e3e2f1333481795a089bef2e24bea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:14:12.313844 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9533060-e083-4875-8639-2707f9e222bc service cinder] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:14:12.314124 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9533060-e083-4875-8639-2707f9e222bc service cinder] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:14:12.314396 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9533060-e083-4875-8639-2707f9e222bc service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:14:12.325433 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9533060-e083-4875-8639-2707f9e222bc service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:14:12.326942 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d9533060-e083-4875-8639-2707f9e222bc service cinder] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:14:12.337513 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1044/2094] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jul 13 22:14:12 2022] GET /identity/v3/auth/tokens => generated 3305 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 22:14:12.750239 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce79401-1c8c-47bb-bf4f-99b9a68a90d4 None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:14:12.812916 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bce79401-1c8c-47bb-bf4f-99b9a68a90d4 None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:14:12.817923 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0219fc78-bc50-4e03-a0ff-5906b9791f7e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:14:12.818820 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0219fc78-bc50-4e03-a0ff-5906b9791f7e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:14:12.819097 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0219fc78-bc50-4e03-a0ff-5906b9791f7e admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:14:12.819365 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0219fc78-bc50-4e03-a0ff-5906b9791f7e admin admin] PATH_INFO: `/v3/users/b49852268c87452bb8a35116c5185e94` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:14:12.820612 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0219fc78-bc50-4e03-a0ff-5906b9791f7e admin admin] RBAC: Authorizing `identity:delete_user(user_id=b49852268c87452bb8a35116c5185e94)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:14:12.822966 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0219fc78-bc50-4e03-a0ff-5906b9791f7e admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:14:12.875266 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-0219fc78-bc50-4e03-a0ff-5906b9791f7e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b49852268c87452bb8a35116c5185e94'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:14:12.889898 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-0219fc78-bc50-4e03-a0ff-5906b9791f7e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b49852268c87452bb8a35116c5185e94'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:14:12.894929 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:14:12.906345 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-0219fc78-bc50-4e03-a0ff-5906b9791f7e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b49852268c87452bb8a35116c5185e94'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:14:12.907403 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-0219fc78-bc50-4e03-a0ff-5906b9791f7e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b49852268c87452bb8a35116c5185e94'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:14:12.908306 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-0219fc78-bc50-4e03-a0ff-5906b9791f7e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b49852268c87452bb8a35116c5185e94'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:14:12.921134 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1051/2095] 10.222.0.52 () {58 vars in 1223 bytes} [Wed Jul 13 22:14:12 2022] DELETE /identity/v3/users/b49852268c87452bb8a35116c5185e94 => generated 0 bytes in 171 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 22:14:12.925718 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae05c70-af7c-48a3-a734-0c1cb4ad300b None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:14:12.926205 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae05c70-af7c-48a3-a734-0c1cb4ad300b None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:14:12.926296 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eae05c70-af7c-48a3-a734-0c1cb4ad300b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:14:12.945780 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-eae05c70-af7c-48a3-a734-0c1cb4ad300b None None] MFA Rules not processed for user `cc99c93da5074cf5a48bc2aa9a92c5e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 22:14:12.947391 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-eae05c70-af7c-48a3-a734-0c1cb4ad300b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:14:12.957902 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:14:13.016924 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1045/2096] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jul 13 22:14:12 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 22:14:13.028178 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0219fc78-bc50-4e03-a0ff-5906b9791f7e admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:14:13.029767 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-0219fc78-bc50-4e03-a0ff-5906b9791f7e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:14:13.088615 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0219fc78-bc50-4e03-a0ff-5906b9791f7e admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:14:13.092706 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19d5ca6-1108-449e-8248-e029abf1dc51 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '847ec9a1bd4d4dec804318a5c845895a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417e3e2f1333481795a089bef2e24bea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:14:13.093373 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19d5ca6-1108-449e-8248-e029abf1dc51 service neutron] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:14:13.093655 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19d5ca6-1108-449e-8248-e029abf1dc51 service neutron] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:14:13.093958 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e19d5ca6-1108-449e-8248-e029abf1dc51 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:14:13.104057 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19d5ca6-1108-449e-8248-e029abf1dc51 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:14:13.105516 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e19d5ca6-1108-449e-8248-e029abf1dc51 service neutron] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:14:13.115295 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1052/2097] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jul 13 22:14:13 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 22:14:13.391569 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae05c70-af7c-48a3-a734-0c1cb4ad300b None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:14:13.393097 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-eae05c70-af7c-48a3-a734-0c1cb4ad300b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:14:13.406128 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eae05c70-af7c-48a3-a734-0c1cb4ad300b None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:14:13.411782 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab36d545-0498-4549-9578-4e3761c7f096 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:14:13.412686 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab36d545-0498-4549-9578-4e3761c7f096 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:14:13.412965 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab36d545-0498-4549-9578-4e3761c7f096 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:14:13.413244 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ab36d545-0498-4549-9578-4e3761c7f096 admin admin] PATH_INFO: `/v3/projects/3c17de2a479e4c4995e71fef83345796` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:14:13.414827 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab36d545-0498-4549-9578-4e3761c7f096 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3c17de2a479e4c4995e71fef83345796)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:14:13.417481 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ab36d545-0498-4549-9578-4e3761c7f096 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:14:13.473756 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-ab36d545-0498-4549-9578-4e3761c7f096 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3c17de2a479e4c4995e71fef83345796', 'request_id': 'req-ab36d545-0498-4549-9578-4e3761c7f096'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:14:13.479924 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-ab36d545-0498-4549-9578-4e3761c7f096 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3c17de2a479e4c4995e71fef83345796', 'request_id': 'req-ab36d545-0498-4549-9578-4e3761c7f096'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:14:13.485746 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-ab36d545-0498-4549-9578-4e3761c7f096 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3c17de2a479e4c4995e71fef83345796', 'request_id': 'req-ab36d545-0498-4549-9578-4e3761c7f096'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:14:13.504956 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-ab36d545-0498-4549-9578-4e3761c7f096 admin admin] The token cache is being invalidate because project 3c17de2a479e4c4995e71fef83345796 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80382) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 22:14:13.505238 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-ab36d545-0498-4549-9578-4e3761c7f096 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:14:13.505870 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-ab36d545-0498-4549-9578-4e3761c7f096 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:14:13.507516 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1046/2098] 10.222.0.52 () {58 vars in 1232 bytes} [Wed Jul 13 22:14:13 2022] DELETE /identity/v3/projects/3c17de2a479e4c4995e71fef83345796 => generated 0 bytes in 117 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 22:14:13.512650 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19d5ca6-1108-449e-8248-e029abf1dc51 service neutron] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:14:13.522456 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e19d5ca6-1108-449e-8248-e029abf1dc51 service neutron] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:14:13.579325 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba4e39e4-0a84-4c13-beb6-20be9452693f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:14:13.580218 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4e39e4-0a84-4c13-beb6-20be9452693f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:14:13.580494 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4e39e4-0a84-4c13-beb6-20be9452693f admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:14:13.580768 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba4e39e4-0a84-4c13-beb6-20be9452693f admin admin] PATH_INFO: `/v3/users/fafca19dfa4640529c12740e7853639a` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:14:13.581979 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4e39e4-0a84-4c13-beb6-20be9452693f admin admin] RBAC: Authorizing `identity:delete_user(user_id=fafca19dfa4640529c12740e7853639a)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:14:13.583970 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba4e39e4-0a84-4c13-beb6-20be9452693f admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:14:13.634901 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-ba4e39e4-0a84-4c13-beb6-20be9452693f admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'fafca19dfa4640529c12740e7853639a'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:14:13.645219 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-ba4e39e4-0a84-4c13-beb6-20be9452693f admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'fafca19dfa4640529c12740e7853639a'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:14:13.659706 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-ba4e39e4-0a84-4c13-beb6-20be9452693f admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'fafca19dfa4640529c12740e7853639a'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:14:13.660495 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-ba4e39e4-0a84-4c13-beb6-20be9452693f admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'fafca19dfa4640529c12740e7853639a'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:14:13.661130 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-ba4e39e4-0a84-4c13-beb6-20be9452693f admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'fafca19dfa4640529c12740e7853639a'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:14:13.670102 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:DELETE=12,keystone:SELECT=87,keystone:INSERT=1 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:14:13.675070 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1053/2099] 10.222.0.52 () {58 vars in 1223 bytes} [Wed Jul 13 22:14:13 2022] DELETE /identity/v3/users/fafca19dfa4640529c12740e7853639a => generated 0 bytes in 163 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 22:14:13.947696 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab36d545-0498-4549-9578-4e3761c7f096 admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:14:13.949234 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-ab36d545-0498-4549-9578-4e3761c7f096 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:14:13.962270 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ab36d545-0498-4549-9578-4e3761c7f096 admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:14:14.019268 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2fdf0e11-4670-46a2-a8fa-56fe755bf77f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:14:14.020127 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdf0e11-4670-46a2-a8fa-56fe755bf77f admin admin] REQUEST_METHOD: `DELETE` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:14:14.020403 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdf0e11-4670-46a2-a8fa-56fe755bf77f admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:14:14.020671 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2fdf0e11-4670-46a2-a8fa-56fe755bf77f admin admin] PATH_INFO: `/v3/projects/a19b4e202eec402c9c6283afe2865a3a` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:14:14.021888 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fdf0e11-4670-46a2-a8fa-56fe755bf77f admin admin] RBAC: Authorizing `identity:delete_project(project_id=a19b4e202eec402c9c6283afe2865a3a)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:14:14.024095 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2fdf0e11-4670-46a2-a8fa-56fe755bf77f admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:14:14.064710 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:14:14.071818 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-2fdf0e11-4670-46a2-a8fa-56fe755bf77f admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a19b4e202eec402c9c6283afe2865a3a', 'request_id': 'req-2fdf0e11-4670-46a2-a8fa-56fe755bf77f'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:14:14.079874 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-2fdf0e11-4670-46a2-a8fa-56fe755bf77f admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a19b4e202eec402c9c6283afe2865a3a', 'request_id': 'req-2fdf0e11-4670-46a2-a8fa-56fe755bf77f'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:14:14.085637 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-2fdf0e11-4670-46a2-a8fa-56fe755bf77f admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a19b4e202eec402c9c6283afe2865a3a', 'request_id': 'req-2fdf0e11-4670-46a2-a8fa-56fe755bf77f'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:14:14.099219 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-2fdf0e11-4670-46a2-a8fa-56fe755bf77f admin admin] The token cache is being invalidate because project a19b4e202eec402c9c6283afe2865a3a was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80382) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 22:14:14.099520 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-2fdf0e11-4670-46a2-a8fa-56fe755bf77f admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:14:14.100153 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-2fdf0e11-4670-46a2-a8fa-56fe755bf77f admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:14:14.101974 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1047/2100] 10.222.0.52 () {58 vars in 1232 bytes} [Wed Jul 13 22:14:13 2022] DELETE /identity/v3/projects/a19b4e202eec402c9c6283afe2865a3a => generated 0 bytes in 155 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 22:14:14.106807 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9eef136-8696-4501-8630-8482b163f0aa None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:14:14.107274 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9eef136-8696-4501-8630-8482b163f0aa None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:14:14.107399 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9eef136-8696-4501-8630-8482b163f0aa None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:14:14.125413 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-c9eef136-8696-4501-8630-8482b163f0aa None None] MFA Rules not processed for user `cc99c93da5074cf5a48bc2aa9a92c5e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 22:14:14.126640 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-c9eef136-8696-4501-8630-8482b163f0aa None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:14:14.185044 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1054/2101] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jul 13 22:14:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 22:14:14.192949 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dafe3d9-b312-4e51-af74-98271d4195e8 None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:14:14.193624 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dafe3d9-b312-4e51-af74-98271d4195e8 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:14:14.194041 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dafe3d9-b312-4e51-af74-98271d4195e8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:14:14.213700 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-7dafe3d9-b312-4e51-af74-98271d4195e8 None None] MFA Rules not processed for user `cc99c93da5074cf5a48bc2aa9a92c5e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 22:14:14.214984 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-7dafe3d9-b312-4e51-af74-98271d4195e8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:14:14.223573 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1048/2102] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jul 13 22:14:14 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 22:14:14.228664 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9eef136-8696-4501-8630-8482b163f0aa None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:14:14.238291 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9eef136-8696-4501-8630-8482b163f0aa None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:14:14.243313 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b9eb04-4a94-4ba9-ab0e-1f320dbc5cf7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:14:14.244063 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b9eb04-4a94-4ba9-ab0e-1f320dbc5cf7 admin admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:14:14.244398 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b9eb04-4a94-4ba9-ab0e-1f320dbc5cf7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:14:14.244734 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b9eb04-4a94-4ba9-ab0e-1f320dbc5cf7 admin admin] PATH_INFO: `/v3/domains` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:14:14.245538 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b9eb04-4a94-4ba9-ab0e-1f320dbc5cf7 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 22:14:14.245937 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b9eb04-4a94-4ba9-ab0e-1f320dbc5cf7 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:14:14.247564 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b9eb04-4a94-4ba9-ab0e-1f320dbc5cf7 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:14:14.259384 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1055/2103] 10.222.0.52 () {58 vars in 1152 bytes} [Wed Jul 13 22:14:14 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 22:14:14.263784 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dafe3d9-b312-4e51-af74-98271d4195e8 None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:14:14.274341 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dafe3d9-b312-4e51-af74-98271d4195e8 None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:14:14.279356 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960e24fc-be9c-4ddb-8b0d-a700ec5aa0d4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:14:14.280016 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960e24fc-be9c-4ddb-8b0d-a700ec5aa0d4 admin admin] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:14:14.280294 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960e24fc-be9c-4ddb-8b0d-a700ec5aa0d4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:14:14.280567 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-960e24fc-be9c-4ddb-8b0d-a700ec5aa0d4 admin admin] PATH_INFO: `/v3/projects` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:14:14.281377 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960e24fc-be9c-4ddb-8b0d-a700ec5aa0d4 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:14:14.283324 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-960e24fc-be9c-4ddb-8b0d-a700ec5aa0d4 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:14:14.311791 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1049/2104] 10.222.0.52 () {60 vars in 1152 bytes} [Wed Jul 13 22:14:14 2022] POST /identity/v3/projects => generated 357 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 13 22:14:14.316542 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b9eb04-4a94-4ba9-ab0e-1f320dbc5cf7 admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:14:14.327540 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b9eb04-4a94-4ba9-ab0e-1f320dbc5cf7 admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:14:14.386966 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b79357f-ee81-4352-810c-53ed2724e7c6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:14:14.387648 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b79357f-ee81-4352-810c-53ed2724e7c6 admin admin] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:14:14.387928 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b79357f-ee81-4352-810c-53ed2724e7c6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:14:14.388203 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b79357f-ee81-4352-810c-53ed2724e7c6 admin admin] PATH_INFO: `/v3/users` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:14:14.388953 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b79357f-ee81-4352-810c-53ed2724e7c6 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:14:14.390977 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b79357f-ee81-4352-810c-53ed2724e7c6 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:14:14.412129 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1056/2105] 10.222.0.52 () {60 vars in 1143 bytes} [Wed Jul 13 22:14:14 2022] POST /identity/v3/users => generated 279 bytes in 96 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 22:14:14.416443 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960e24fc-be9c-4ddb-8b0d-a700ec5aa0d4 admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:14:14.431281 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-960e24fc-be9c-4ddb-8b0d-a700ec5aa0d4 admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:14:14.437306 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c355e6f4-a60b-44fa-a07f-217ab0dbb3cd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:14:14.437978 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c355e6f4-a60b-44fa-a07f-217ab0dbb3cd admin admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:14:14.438259 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c355e6f4-a60b-44fa-a07f-217ab0dbb3cd admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:14:14.438558 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c355e6f4-a60b-44fa-a07f-217ab0dbb3cd admin admin] PATH_INFO: `/v3/roles` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:14:14.439300 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c355e6f4-a60b-44fa-a07f-217ab0dbb3cd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:14:14.440955 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c355e6f4-a60b-44fa-a07f-217ab0dbb3cd admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:14:14.452805 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1050/2106] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jul 13 22:14:14 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 13 22:14:14.456777 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b79357f-ee81-4352-810c-53ed2724e7c6 admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:14:14.467825 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b79357f-ee81-4352-810c-53ed2724e7c6 admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:14:14.472965 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-da0b3c8b-d28e-4d8c-8cc3-2eb99153f288 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:14:14.473624 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0b3c8b-d28e-4d8c-8cc3-2eb99153f288 admin admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:14:14.473902 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0b3c8b-d28e-4d8c-8cc3-2eb99153f288 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:14:14.474158 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-da0b3c8b-d28e-4d8c-8cc3-2eb99153f288 admin admin] PATH_INFO: `/v3/roles` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:14:14.474944 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0b3c8b-d28e-4d8c-8cc3-2eb99153f288 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:14:14.477177 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-da0b3c8b-d28e-4d8c-8cc3-2eb99153f288 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:14:14.488449 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1057/2107] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jul 13 22:14:14 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 22:14:14.494006 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c355e6f4-a60b-44fa-a07f-217ab0dbb3cd admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:14:14.504434 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c355e6f4-a60b-44fa-a07f-217ab0dbb3cd admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:14:14.510228 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1b3a54-7004-4818-a00e-105ea151eb92 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:14:14.511174 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1b3a54-7004-4818-a00e-105ea151eb92 admin admin] REQUEST_METHOD: `PUT` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:14:14.511475 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1b3a54-7004-4818-a00e-105ea151eb92 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:14:14.511726 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-af1b3a54-7004-4818-a00e-105ea151eb92 admin admin] PATH_INFO: `/v3/projects/c07acf7fbbd74093bbe5cf32950b449c/users/f692077bec1744f4b1dddd3d831975b7/roles/2a5e86976394498abb6310697cdb3f8b` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:14:14.527165 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1b3a54-7004-4818-a00e-105ea151eb92 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c07acf7fbbd74093bbe5cf32950b449c, user_id=f692077bec1744f4b1dddd3d831975b7, role_id=2a5e86976394498abb6310697cdb3f8b)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:14:14.529860 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-af1b3a54-7004-4818-a00e-105ea151eb92 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:14:14.544932 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1051/2108] 10.222.0.52 () {60 vars in 1482 bytes} [Wed Jul 13 22:14:14 2022] PUT /identity/v3/projects/c07acf7fbbd74093bbe5cf32950b449c/users/f692077bec1744f4b1dddd3d831975b7/roles/2a5e86976394498abb6310697cdb3f8b => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 13 22:14:14.555333 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5518462e-edf2-43e2-9da4-a48004f6ca59 None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:14:14.555623 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5518462e-edf2-43e2-9da4-a48004f6ca59 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:14:14.555903 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5518462e-edf2-43e2-9da4-a48004f6ca59 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:14:14.583532 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-5518462e-edf2-43e2-9da4-a48004f6ca59 None None] MFA Rules not processed for user `f692077bec1744f4b1dddd3d831975b7`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 22:14:14.585204 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-5518462e-edf2-43e2-9da4-a48004f6ca59 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:14:14.648018 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:14:14.659846 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1058/2109] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jul 13 22:14:14 2022] POST /identity/v3/auth/tokens => generated 3224 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 22:14:14.664314 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1b3a54-7004-4818-a00e-105ea151eb92 admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:14:14.728670 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-af1b3a54-7004-4818-a00e-105ea151eb92 admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:14:14.733644 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6550c106-3fe3-4be6-9a96-4174b3c1d4f5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:14:14.734302 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6550c106-3fe3-4be6-9a96-4174b3c1d4f5 admin admin] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:14:14.734616 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6550c106-3fe3-4be6-9a96-4174b3c1d4f5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:14:14.734892 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6550c106-3fe3-4be6-9a96-4174b3c1d4f5 admin admin] PATH_INFO: `/v3/projects` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:14:14.735588 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6550c106-3fe3-4be6-9a96-4174b3c1d4f5 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:14:14.737500 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6550c106-3fe3-4be6-9a96-4174b3c1d4f5 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:14:14.767467 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1052/2110] 10.222.0.52 () {60 vars in 1152 bytes} [Wed Jul 13 22:14:14 2022] POST /identity/v3/projects => generated 355 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 22:14:14.771848 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5518462e-edf2-43e2-9da4-a48004f6ca59 None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:14:14.782989 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5518462e-edf2-43e2-9da4-a48004f6ca59 None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:14:14.842620 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371cf255-70e3-4dec-b20f-a69aab5a5c45 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:14:14.843725 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371cf255-70e3-4dec-b20f-a69aab5a5c45 admin admin] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:14:14.844006 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371cf255-70e3-4dec-b20f-a69aab5a5c45 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:14:14.844279 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-371cf255-70e3-4dec-b20f-a69aab5a5c45 admin admin] PATH_INFO: `/v3/users` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:14:14.844991 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371cf255-70e3-4dec-b20f-a69aab5a5c45 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:14:14.847588 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-371cf255-70e3-4dec-b20f-a69aab5a5c45 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:14:14.870512 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1059/2111] 10.222.0.52 () {60 vars in 1143 bytes} [Wed Jul 13 22:14:14 2022] POST /identity/v3/users => generated 284 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 13 22:14:14.874745 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6550c106-3fe3-4be6-9a96-4174b3c1d4f5 admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:14:14.885625 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6550c106-3fe3-4be6-9a96-4174b3c1d4f5 admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:14:14.890536 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d00a19e-0ce0-4a2f-8e52-6efcd63cf162 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:14:14.891257 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d00a19e-0ce0-4a2f-8e52-6efcd63cf162 admin admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:14:14.891534 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d00a19e-0ce0-4a2f-8e52-6efcd63cf162 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:14:14.891807 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8d00a19e-0ce0-4a2f-8e52-6efcd63cf162 admin admin] PATH_INFO: `/v3/roles` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:14:14.892539 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d00a19e-0ce0-4a2f-8e52-6efcd63cf162 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:14:14.894153 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8d00a19e-0ce0-4a2f-8e52-6efcd63cf162 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:14:14.905244 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1053/2112] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jul 13 22:14:14 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 22:14:14.909553 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371cf255-70e3-4dec-b20f-a69aab5a5c45 admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:14:14.920492 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-371cf255-70e3-4dec-b20f-a69aab5a5c45 admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:14:14.925768 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502e2e5c-5424-4ce9-9753-3ccb033ac753 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:14:14.926708 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502e2e5c-5424-4ce9-9753-3ccb033ac753 admin admin] REQUEST_METHOD: `PUT` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:14:14.926997 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502e2e5c-5424-4ce9-9753-3ccb033ac753 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:14:14.927274 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-502e2e5c-5424-4ce9-9753-3ccb033ac753 admin admin] PATH_INFO: `/v3/projects/41ee264c0c5f40418538af935735e6bb/users/014811d3e7a64293b2f854ec8a169636/roles/44efba2579f342df8a0789c59d163ec3` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:14:14.942989 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502e2e5c-5424-4ce9-9753-3ccb033ac753 admin admin] RBAC: Authorizing `identity:create_grant(project_id=41ee264c0c5f40418538af935735e6bb, user_id=014811d3e7a64293b2f854ec8a169636, role_id=44efba2579f342df8a0789c59d163ec3)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:14:14.945894 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-502e2e5c-5424-4ce9-9753-3ccb033ac753 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:14:14.962576 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1060/2113] 10.222.0.52 () {60 vars in 1482 bytes} [Wed Jul 13 22:14:14 2022] PUT /identity/v3/projects/41ee264c0c5f40418538af935735e6bb/users/014811d3e7a64293b2f854ec8a169636/roles/44efba2579f342df8a0789c59d163ec3 => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 22:14:14.966568 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d00a19e-0ce0-4a2f-8e52-6efcd63cf162 admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:14:14.985864 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8d00a19e-0ce0-4a2f-8e52-6efcd63cf162 admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:14:15.030577 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:14:15.055707 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-96ea570c-d1b4-46d1-8d61-5cb0271cd3b3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:14:15.056397 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ea570c-d1b4-46d1-8d61-5cb0271cd3b3 admin admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:14:15.056677 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ea570c-d1b4-46d1-8d61-5cb0271cd3b3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:14:15.056953 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-96ea570c-d1b4-46d1-8d61-5cb0271cd3b3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:14:15.057695 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ea570c-d1b4-46d1-8d61-5cb0271cd3b3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:14:15.059398 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-96ea570c-d1b4-46d1-8d61-5cb0271cd3b3 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:14:15.070971 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1054/2114] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jul 13 22:14:14 2022] GET /identity/v3/roles => generated 1420 bytes in 105 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 13 22:14:15.075502 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502e2e5c-5424-4ce9-9753-3ccb033ac753 admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:14:15.087181 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-502e2e5c-5424-4ce9-9753-3ccb033ac753 admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:14:15.092051 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3226294-111b-4c54-86bb-1ae2aad2f115 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:14:15.093108 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3226294-111b-4c54-86bb-1ae2aad2f115 admin admin] REQUEST_METHOD: `PUT` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:14:15.093400 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3226294-111b-4c54-86bb-1ae2aad2f115 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:14:15.093733 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d3226294-111b-4c54-86bb-1ae2aad2f115 admin admin] PATH_INFO: `/v3/projects/41ee264c0c5f40418538af935735e6bb/users/014811d3e7a64293b2f854ec8a169636/roles/44efba2579f342df8a0789c59d163ec3` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:14:15.095963 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3226294-111b-4c54-86bb-1ae2aad2f115 admin admin] RBAC: Authorizing `identity:create_grant(project_id=41ee264c0c5f40418538af935735e6bb, user_id=014811d3e7a64293b2f854ec8a169636, role_id=44efba2579f342df8a0789c59d163ec3)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:14:15.098741 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d3226294-111b-4c54-86bb-1ae2aad2f115 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:14:15.110462 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1061/2115] 10.222.0.52 () {60 vars in 1482 bytes} [Wed Jul 13 22:14:15 2022] PUT /identity/v3/projects/41ee264c0c5f40418538af935735e6bb/users/014811d3e7a64293b2f854ec8a169636/roles/44efba2579f342df8a0789c59d163ec3 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 22:14:15.121939 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a70afd-94cc-4460-9e62-1225019084d1 None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:14:15.122227 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a70afd-94cc-4460-9e62-1225019084d1 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:14:15.122554 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-84a70afd-94cc-4460-9e62-1225019084d1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:14:15.151813 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-84a70afd-94cc-4460-9e62-1225019084d1 None None] MFA Rules not processed for user `014811d3e7a64293b2f854ec8a169636`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 22:14:15.153038 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-84a70afd-94cc-4460-9e62-1225019084d1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:14:15.226397 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1055/2116] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jul 13 22:14:15 2022] POST /identity/v3/auth/tokens => generated 3289 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 22:14:15.237812 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3226294-111b-4c54-86bb-1ae2aad2f115 admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:14:15.239562 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-d3226294-111b-4c54-86bb-1ae2aad2f115 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:14:15.298268 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d3226294-111b-4c54-86bb-1ae2aad2f115 admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:14:15.302993 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ee5f35-5fb7-495c-b04e-7390ef0dae3d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc77e1ea1ada4700b809beb1c980c5e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417e3e2f1333481795a089bef2e24bea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:14:15.303977 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ee5f35-5fb7-495c-b04e-7390ef0dae3d service cinder] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:14:15.304294 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ee5f35-5fb7-495c-b04e-7390ef0dae3d service cinder] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:14:15.304603 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0ee5f35-5fb7-495c-b04e-7390ef0dae3d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:14:15.314054 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ee5f35-5fb7-495c-b04e-7390ef0dae3d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:14:15.315529 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0ee5f35-5fb7-495c-b04e-7390ef0dae3d service cinder] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:14:15.325607 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1062/2117] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jul 13 22:14:15 2022] GET /identity/v3/auth/tokens => generated 3289 bytes in 88 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 22:14:15.399515 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a70afd-94cc-4460-9e62-1225019084d1 None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:14:15.410385 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-84a70afd-94cc-4460-9e62-1225019084d1 None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:14:15.414224 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dcfd1a0-6134-4cc1-8c71-d65212aab1e0 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc77e1ea1ada4700b809beb1c980c5e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417e3e2f1333481795a089bef2e24bea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:14:15.414952 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcfd1a0-6134-4cc1-8c71-d65212aab1e0 service cinder] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:14:15.415238 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcfd1a0-6134-4cc1-8c71-d65212aab1e0 service cinder] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:14:15.415509 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dcfd1a0-6134-4cc1-8c71-d65212aab1e0 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:14:15.477201 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dcfd1a0-6134-4cc1-8c71-d65212aab1e0 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:14:15.478671 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dcfd1a0-6134-4cc1-8c71-d65212aab1e0 service cinder] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:14:15.488392 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1056/2118] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jul 13 22:14:15 2022] GET /identity/v3/auth/tokens => generated 3224 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 22:14:25.320856 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=73,keystone:INSERT=5 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:14:25.483805 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=73 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:14:30.550550 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ee5f35-5fb7-495c-b04e-7390ef0dae3d service cinder] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:14:30.621246 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0ee5f35-5fb7-495c-b04e-7390ef0dae3d service cinder] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:14:30.625762 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a5ac00-eed2-4f3f-a95a-bef3780bcc87 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:14:30.626711 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a5ac00-eed2-4f3f-a95a-bef3780bcc87 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:14:30.627053 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a5ac00-eed2-4f3f-a95a-bef3780bcc87 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:14:30.627418 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-89a5ac00-eed2-4f3f-a95a-bef3780bcc87 admin admin] PATH_INFO: `/v3/users/f692077bec1744f4b1dddd3d831975b7` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:14:30.628585 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a5ac00-eed2-4f3f-a95a-bef3780bcc87 admin admin] RBAC: Authorizing `identity:delete_user(user_id=f692077bec1744f4b1dddd3d831975b7)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:14:30.630560 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-89a5ac00-eed2-4f3f-a95a-bef3780bcc87 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:14:30.686762 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-89a5ac00-eed2-4f3f-a95a-bef3780bcc87 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'f692077bec1744f4b1dddd3d831975b7'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:14:30.696538 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-89a5ac00-eed2-4f3f-a95a-bef3780bcc87 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'f692077bec1744f4b1dddd3d831975b7'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:14:30.709090 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-89a5ac00-eed2-4f3f-a95a-bef3780bcc87 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'f692077bec1744f4b1dddd3d831975b7'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:14:30.709726 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-89a5ac00-eed2-4f3f-a95a-bef3780bcc87 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'f692077bec1744f4b1dddd3d831975b7'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:14:30.710313 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-89a5ac00-eed2-4f3f-a95a-bef3780bcc87 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'f692077bec1744f4b1dddd3d831975b7'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:14:30.722658 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1063/2119] 10.222.0.52 () {58 vars in 1223 bytes} [Wed Jul 13 22:14:30 2022] DELETE /identity/v3/users/f692077bec1744f4b1dddd3d831975b7 => generated 0 bytes in 173 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 22:14:30.727398 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77c5802-0cca-407b-a2fc-89719f501179 None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:14:30.727894 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77c5802-0cca-407b-a2fc-89719f501179 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:14:30.728041 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c77c5802-0cca-407b-a2fc-89719f501179 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:14:30.746868 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-c77c5802-0cca-407b-a2fc-89719f501179 None None] MFA Rules not processed for user `cc99c93da5074cf5a48bc2aa9a92c5e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 22:14:30.748181 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-c77c5802-0cca-407b-a2fc-89719f501179 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:14:30.822922 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1057/2120] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jul 13 22:14:30 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 22:14:30.832858 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a5ac00-eed2-4f3f-a95a-bef3780bcc87 admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:14:30.834452 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-89a5ac00-eed2-4f3f-a95a-bef3780bcc87 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:14:30.898739 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-89a5ac00-eed2-4f3f-a95a-bef3780bcc87 admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:14:30.902459 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3000bc7f-6bfa-4ff1-bbbd-dc9b091268e3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '847ec9a1bd4d4dec804318a5c845895a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417e3e2f1333481795a089bef2e24bea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:14:30.903329 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3000bc7f-6bfa-4ff1-bbbd-dc9b091268e3 service neutron] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:14:30.903691 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3000bc7f-6bfa-4ff1-bbbd-dc9b091268e3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:14:30.903875 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3000bc7f-6bfa-4ff1-bbbd-dc9b091268e3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:14:30.913256 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3000bc7f-6bfa-4ff1-bbbd-dc9b091268e3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:14:30.914736 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3000bc7f-6bfa-4ff1-bbbd-dc9b091268e3 service neutron] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:14:30.924355 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1064/2121] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jul 13 22:14:30 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 22:14:31.216674 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77c5802-0cca-407b-a2fc-89719f501179 None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:14:31.218140 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-c77c5802-0cca-407b-a2fc-89719f501179 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:14:31.240321 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c77c5802-0cca-407b-a2fc-89719f501179 None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:14:31.245413 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05df9df8-17f0-4f93-a22b-38bf56bf54c2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:14:31.246342 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05df9df8-17f0-4f93-a22b-38bf56bf54c2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:14:31.246637 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05df9df8-17f0-4f93-a22b-38bf56bf54c2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:14:31.246860 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-05df9df8-17f0-4f93-a22b-38bf56bf54c2 admin admin] PATH_INFO: `/v3/projects/c07acf7fbbd74093bbe5cf32950b449c` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:14:31.248060 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05df9df8-17f0-4f93-a22b-38bf56bf54c2 admin admin] RBAC: Authorizing `identity:delete_project(project_id=c07acf7fbbd74093bbe5cf32950b449c)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:14:31.249977 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-05df9df8-17f0-4f93-a22b-38bf56bf54c2 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:14:31.312464 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-05df9df8-17f0-4f93-a22b-38bf56bf54c2 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c07acf7fbbd74093bbe5cf32950b449c', 'request_id': 'req-05df9df8-17f0-4f93-a22b-38bf56bf54c2'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:14:31.318801 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-05df9df8-17f0-4f93-a22b-38bf56bf54c2 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c07acf7fbbd74093bbe5cf32950b449c', 'request_id': 'req-05df9df8-17f0-4f93-a22b-38bf56bf54c2'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:14:31.325048 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-05df9df8-17f0-4f93-a22b-38bf56bf54c2 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c07acf7fbbd74093bbe5cf32950b449c', 'request_id': 'req-05df9df8-17f0-4f93-a22b-38bf56bf54c2'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:14:31.338702 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-05df9df8-17f0-4f93-a22b-38bf56bf54c2 admin admin] The token cache is being invalidate because project c07acf7fbbd74093bbe5cf32950b449c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80382) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 22:14:31.339092 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-05df9df8-17f0-4f93-a22b-38bf56bf54c2 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:14:31.339812 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-05df9df8-17f0-4f93-a22b-38bf56bf54c2 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:14:31.341738 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1058/2122] 10.222.0.52 () {58 vars in 1232 bytes} [Wed Jul 13 22:14:31 2022] DELETE /identity/v3/projects/c07acf7fbbd74093bbe5cf32950b449c => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 22:14:31.345914 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3000bc7f-6bfa-4ff1-bbbd-dc9b091268e3 service neutron] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:14:31.356193 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3000bc7f-6bfa-4ff1-bbbd-dc9b091268e3 service neutron] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:14:31.375364 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:14:31.420384 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-35c29efe-e11e-4cd5-bd8f-1e38985b3941 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:14:31.421687 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c29efe-e11e-4cd5-bd8f-1e38985b3941 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:14:31.422011 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c29efe-e11e-4cd5-bd8f-1e38985b3941 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:14:31.422351 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-35c29efe-e11e-4cd5-bd8f-1e38985b3941 admin admin] PATH_INFO: `/v3/users/014811d3e7a64293b2f854ec8a169636` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:14:31.424531 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c29efe-e11e-4cd5-bd8f-1e38985b3941 admin admin] RBAC: Authorizing `identity:delete_user(user_id=014811d3e7a64293b2f854ec8a169636)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:14:31.426524 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-35c29efe-e11e-4cd5-bd8f-1e38985b3941 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:14:31.486739 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-35c29efe-e11e-4cd5-bd8f-1e38985b3941 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '014811d3e7a64293b2f854ec8a169636'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:14:31.495810 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-35c29efe-e11e-4cd5-bd8f-1e38985b3941 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '014811d3e7a64293b2f854ec8a169636'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:14:31.508339 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-35c29efe-e11e-4cd5-bd8f-1e38985b3941 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '014811d3e7a64293b2f854ec8a169636'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:14:31.509044 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-35c29efe-e11e-4cd5-bd8f-1e38985b3941 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '014811d3e7a64293b2f854ec8a169636'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:14:31.510006 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-35c29efe-e11e-4cd5-bd8f-1e38985b3941 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '014811d3e7a64293b2f854ec8a169636'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:14:31.533925 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1065/2123] 10.222.0.52 () {58 vars in 1223 bytes} [Wed Jul 13 22:14:31 2022] DELETE /identity/v3/users/014811d3e7a64293b2f854ec8a169636 => generated 0 bytes in 186 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 13 22:14:31.860254 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05df9df8-17f0-4f93-a22b-38bf56bf54c2 admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:14:31.861990 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-05df9df8-17f0-4f93-a22b-38bf56bf54c2 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:14:31.876200 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-05df9df8-17f0-4f93-a22b-38bf56bf54c2 admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:14:31.944989 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff0ed488-d864-48f9-a55a-baaaea8f8384 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:14:31.945896 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0ed488-d864-48f9-a55a-baaaea8f8384 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:14:31.946173 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0ed488-d864-48f9-a55a-baaaea8f8384 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:14:31.946480 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff0ed488-d864-48f9-a55a-baaaea8f8384 admin admin] PATH_INFO: `/v3/projects/41ee264c0c5f40418538af935735e6bb` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:14:31.948237 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0ed488-d864-48f9-a55a-baaaea8f8384 admin admin] RBAC: Authorizing `identity:delete_project(project_id=41ee264c0c5f40418538af935735e6bb)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:14:31.951069 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff0ed488-d864-48f9-a55a-baaaea8f8384 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:14:31.984020 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=91,keystone:DELETE=8,keystone:INSERT=1 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:14:32.001726 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-ff0ed488-d864-48f9-a55a-baaaea8f8384 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '41ee264c0c5f40418538af935735e6bb', 'request_id': 'req-ff0ed488-d864-48f9-a55a-baaaea8f8384'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:14:32.007980 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-ff0ed488-d864-48f9-a55a-baaaea8f8384 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '41ee264c0c5f40418538af935735e6bb', 'request_id': 'req-ff0ed488-d864-48f9-a55a-baaaea8f8384'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:14:32.014093 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-ff0ed488-d864-48f9-a55a-baaaea8f8384 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '41ee264c0c5f40418538af935735e6bb', 'request_id': 'req-ff0ed488-d864-48f9-a55a-baaaea8f8384'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:14:32.031411 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-ff0ed488-d864-48f9-a55a-baaaea8f8384 admin admin] The token cache is being invalidate because project 41ee264c0c5f40418538af935735e6bb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80382) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 22:14:32.031687 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-ff0ed488-d864-48f9-a55a-baaaea8f8384 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:14:32.032356 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-ff0ed488-d864-48f9-a55a-baaaea8f8384 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:14:32.033934 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1059/2124] 10.222.0.52 () {58 vars in 1232 bytes} [Wed Jul 13 22:14:31 2022] DELETE /identity/v3/projects/41ee264c0c5f40418538af935735e6bb => generated 0 bytes in 174 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 22:14:32.039882 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebfd6e9-33ae-4a12-bf65-110f40c7d3de None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:14:32.040416 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebfd6e9-33ae-4a12-bf65-110f40c7d3de None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:14:32.040416 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cebfd6e9-33ae-4a12-bf65-110f40c7d3de None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:14:32.059005 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-cebfd6e9-33ae-4a12-bf65-110f40c7d3de None None] MFA Rules not processed for user `cc99c93da5074cf5a48bc2aa9a92c5e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 22:14:32.060203 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-cebfd6e9-33ae-4a12-bf65-110f40c7d3de None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:14:32.124585 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1066/2125] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jul 13 22:14:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 86 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 13 22:14:32.132855 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8f9100-c172-4b59-80f1-e8616c348025 None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:14:32.133539 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8f9100-c172-4b59-80f1-e8616c348025 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:14:32.133828 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b8f9100-c172-4b59-80f1-e8616c348025 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:14:32.153813 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-1b8f9100-c172-4b59-80f1-e8616c348025 None None] MFA Rules not processed for user `cc99c93da5074cf5a48bc2aa9a92c5e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 22:14:32.155059 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-1b8f9100-c172-4b59-80f1-e8616c348025 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:14:32.164402 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1060/2126] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jul 13 22:14:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 22:14:32.169033 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebfd6e9-33ae-4a12-bf65-110f40c7d3de None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:14:32.182032 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cebfd6e9-33ae-4a12-bf65-110f40c7d3de None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:14:32.189428 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2177e02-9751-41a8-8df7-acb8b4db55bf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:14:32.190139 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2177e02-9751-41a8-8df7-acb8b4db55bf admin admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:14:32.190390 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2177e02-9751-41a8-8df7-acb8b4db55bf admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:14:32.190699 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2177e02-9751-41a8-8df7-acb8b4db55bf admin admin] PATH_INFO: `/v3/domains` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:14:32.191442 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2177e02-9751-41a8-8df7-acb8b4db55bf admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 22:14:32.191743 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2177e02-9751-41a8-8df7-acb8b4db55bf admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:14:32.193367 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2177e02-9751-41a8-8df7-acb8b4db55bf admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:14:32.206883 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1067/2127] 10.222.0.52 () {58 vars in 1152 bytes} [Wed Jul 13 22:14:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 22:14:32.211609 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8f9100-c172-4b59-80f1-e8616c348025 None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:14:32.222359 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b8f9100-c172-4b59-80f1-e8616c348025 None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:14:32.227453 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5ac7d9-9da8-47de-91da-201c16e79daa admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:14:32.228124 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5ac7d9-9da8-47de-91da-201c16e79daa admin admin] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:14:32.228392 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5ac7d9-9da8-47de-91da-201c16e79daa admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:14:32.228665 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a5ac7d9-9da8-47de-91da-201c16e79daa admin admin] PATH_INFO: `/v3/projects` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:14:32.229452 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5ac7d9-9da8-47de-91da-201c16e79daa admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:14:32.231450 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6a5ac7d9-9da8-47de-91da-201c16e79daa admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:14:32.259575 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1061/2128] 10.222.0.52 () {60 vars in 1152 bytes} [Wed Jul 13 22:14:32 2022] POST /identity/v3/projects => generated 401 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 22:14:32.264051 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2177e02-9751-41a8-8df7-acb8b4db55bf admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:14:32.274527 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2177e02-9751-41a8-8df7-acb8b4db55bf admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:14:32.306314 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:14:32.339867 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbec6616-9c1b-4dd3-aa79-3fa511a1463f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:14:32.340549 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbec6616-9c1b-4dd3-aa79-3fa511a1463f admin admin] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:14:32.340830 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbec6616-9c1b-4dd3-aa79-3fa511a1463f admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:14:32.341104 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fbec6616-9c1b-4dd3-aa79-3fa511a1463f admin admin] PATH_INFO: `/v3/users` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:14:32.341812 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbec6616-9c1b-4dd3-aa79-3fa511a1463f admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:14:32.343818 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fbec6616-9c1b-4dd3-aa79-3fa511a1463f admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:14:32.367558 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1068/2129] 10.222.0.52 () {60 vars in 1143 bytes} [Wed Jul 13 22:14:32 2022] POST /identity/v3/users => generated 301 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 13 22:14:32.372218 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5ac7d9-9da8-47de-91da-201c16e79daa admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:14:32.383980 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a5ac7d9-9da8-47de-91da-201c16e79daa admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:14:32.391212 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b171ea5a-345c-4eb6-bf35-5027f7d59829 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:14:32.392019 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b171ea5a-345c-4eb6-bf35-5027f7d59829 admin admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:14:32.392379 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b171ea5a-345c-4eb6-bf35-5027f7d59829 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:14:32.392717 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b171ea5a-345c-4eb6-bf35-5027f7d59829 admin admin] PATH_INFO: `/v3/roles` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:14:32.393535 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b171ea5a-345c-4eb6-bf35-5027f7d59829 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:14:32.395277 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b171ea5a-345c-4eb6-bf35-5027f7d59829 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:14:32.407638 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1062/2130] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jul 13 22:14:32 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 22:14:32.411577 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbec6616-9c1b-4dd3-aa79-3fa511a1463f admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:14:32.421729 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fbec6616-9c1b-4dd3-aa79-3fa511a1463f admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:14:32.426514 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-06a69fed-df18-4df3-b2d7-7d844ca95c5c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:14:32.427196 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a69fed-df18-4df3-b2d7-7d844ca95c5c admin admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:14:32.427474 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a69fed-df18-4df3-b2d7-7d844ca95c5c admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:14:32.427743 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-06a69fed-df18-4df3-b2d7-7d844ca95c5c admin admin] PATH_INFO: `/v3/roles` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:14:32.428478 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a69fed-df18-4df3-b2d7-7d844ca95c5c admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:14:32.430051 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-06a69fed-df18-4df3-b2d7-7d844ca95c5c admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:14:32.440427 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1069/2131] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jul 13 22:14:32 2022] GET /identity/v3/roles => generated 1420 bytes in 29 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 22:14:32.444634 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b171ea5a-345c-4eb6-bf35-5027f7d59829 admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:14:32.455313 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b171ea5a-345c-4eb6-bf35-5027f7d59829 admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:14:32.460990 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7dba54-d07a-4edb-86f0-b8b1389f4df6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:14:32.461821 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7dba54-d07a-4edb-86f0-b8b1389f4df6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:14:32.462094 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7dba54-d07a-4edb-86f0-b8b1389f4df6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:14:32.462395 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ba7dba54-d07a-4edb-86f0-b8b1389f4df6 admin admin] PATH_INFO: `/v3/projects/ed432eceba0645e8a1039f65b1a45e8d/users/5d5cd89b7edb4eadbdfec0dba1e2dee8/roles/2a5e86976394498abb6310697cdb3f8b` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:14:32.478069 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7dba54-d07a-4edb-86f0-b8b1389f4df6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ed432eceba0645e8a1039f65b1a45e8d, user_id=5d5cd89b7edb4eadbdfec0dba1e2dee8, role_id=2a5e86976394498abb6310697cdb3f8b)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:14:32.480783 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ba7dba54-d07a-4edb-86f0-b8b1389f4df6 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:14:32.512129 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1063/2132] 10.222.0.52 () {60 vars in 1482 bytes} [Wed Jul 13 22:14:32 2022] PUT /identity/v3/projects/ed432eceba0645e8a1039f65b1a45e8d/users/5d5cd89b7edb4eadbdfec0dba1e2dee8/roles/2a5e86976394498abb6310697cdb3f8b => generated 0 bytes in 68 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 13 22:14:32.523888 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26d43ea-50fe-418f-86c7-60ac4006a9b4 None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:14:32.524175 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26d43ea-50fe-418f-86c7-60ac4006a9b4 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:14:32.524448 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b26d43ea-50fe-418f-86c7-60ac4006a9b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:14:32.559444 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-b26d43ea-50fe-418f-86c7-60ac4006a9b4 None None] MFA Rules not processed for user `5d5cd89b7edb4eadbdfec0dba1e2dee8`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 22:14:32.560682 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-b26d43ea-50fe-418f-86c7-60ac4006a9b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:14:32.633279 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1070/2133] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jul 13 22:14:32 2022] POST /identity/v3/auth/tokens => generated 3268 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 22:14:32.637949 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7dba54-d07a-4edb-86f0-b8b1389f4df6 admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:14:32.713941 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ba7dba54-d07a-4edb-86f0-b8b1389f4df6 admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:14:32.720085 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155e7ca3-310d-4a3d-9bd6-8aaf43aa2ceb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:14:32.720806 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155e7ca3-310d-4a3d-9bd6-8aaf43aa2ceb admin admin] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:14:32.721048 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155e7ca3-310d-4a3d-9bd6-8aaf43aa2ceb admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:14:32.721374 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-155e7ca3-310d-4a3d-9bd6-8aaf43aa2ceb admin admin] PATH_INFO: `/v3/projects` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:14:32.722196 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-155e7ca3-310d-4a3d-9bd6-8aaf43aa2ceb admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:14:32.724375 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-155e7ca3-310d-4a3d-9bd6-8aaf43aa2ceb admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:14:32.753994 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1064/2134] 10.222.0.52 () {60 vars in 1152 bytes} [Wed Jul 13 22:14:32 2022] POST /identity/v3/projects => generated 399 bytes in 117 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 13 22:14:32.758418 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26d43ea-50fe-418f-86c7-60ac4006a9b4 None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:14:32.769962 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b26d43ea-50fe-418f-86c7-60ac4006a9b4 None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:14:32.830415 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae228e8-1b0a-40de-9f04-cc7d2fc91e57 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:14:32.831134 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae228e8-1b0a-40de-9f04-cc7d2fc91e57 admin admin] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:14:32.831414 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae228e8-1b0a-40de-9f04-cc7d2fc91e57 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:14:32.831685 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ae228e8-1b0a-40de-9f04-cc7d2fc91e57 admin admin] PATH_INFO: `/v3/users` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:14:32.832391 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae228e8-1b0a-40de-9f04-cc7d2fc91e57 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:14:32.834430 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ae228e8-1b0a-40de-9f04-cc7d2fc91e57 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:14:32.861294 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1071/2135] 10.222.0.52 () {60 vars in 1143 bytes} [Wed Jul 13 22:14:32 2022] POST /identity/v3/users => generated 306 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 22:14:32.862461 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155e7ca3-310d-4a3d-9bd6-8aaf43aa2ceb admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:14:32.874040 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-155e7ca3-310d-4a3d-9bd6-8aaf43aa2ceb admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:14:32.879358 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6860fcaf-295f-4c76-9515-bed11f5330e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:14:32.880030 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6860fcaf-295f-4c76-9515-bed11f5330e2 admin admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:14:32.880308 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6860fcaf-295f-4c76-9515-bed11f5330e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:14:32.880607 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6860fcaf-295f-4c76-9515-bed11f5330e2 admin admin] PATH_INFO: `/v3/roles` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:14:32.881336 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6860fcaf-295f-4c76-9515-bed11f5330e2 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:14:32.882994 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6860fcaf-295f-4c76-9515-bed11f5330e2 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:14:32.895241 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1065/2136] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jul 13 22:14:32 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 22:14:32.900038 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae228e8-1b0a-40de-9f04-cc7d2fc91e57 admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:14:32.911723 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ae228e8-1b0a-40de-9f04-cc7d2fc91e57 admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:14:32.917776 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dfc2724-71a6-48a8-8913-668ec14d8fd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:14:32.918717 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfc2724-71a6-48a8-8913-668ec14d8fd3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:14:32.918988 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfc2724-71a6-48a8-8913-668ec14d8fd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:14:32.919275 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7dfc2724-71a6-48a8-8913-668ec14d8fd3 admin admin] PATH_INFO: `/v3/projects/6090275139944ca28ea6fcf64b8b5aeb/users/7834297c19ec4593a9626d5fecd60fd0/roles/44efba2579f342df8a0789c59d163ec3` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:14:32.937626 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dfc2724-71a6-48a8-8913-668ec14d8fd3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6090275139944ca28ea6fcf64b8b5aeb, user_id=7834297c19ec4593a9626d5fecd60fd0, role_id=44efba2579f342df8a0789c59d163ec3)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:14:32.940388 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7dfc2724-71a6-48a8-8913-668ec14d8fd3 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:14:32.955771 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1072/2137] 10.222.0.52 () {60 vars in 1482 bytes} [Wed Jul 13 22:14:32 2022] PUT /identity/v3/projects/6090275139944ca28ea6fcf64b8b5aeb/users/7834297c19ec4593a9626d5fecd60fd0/roles/44efba2579f342df8a0789c59d163ec3 => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 13 22:14:32.959914 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6860fcaf-295f-4c76-9515-bed11f5330e2 admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:14:32.970964 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6860fcaf-295f-4c76-9515-bed11f5330e2 admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:14:32.995897 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:DELETE=6,keystone:SELECT=90,keystone:INSERT=4 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:14:33.037964 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3c033bf8-a87f-4794-83f6-6a8258908351 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:14:33.038664 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c033bf8-a87f-4794-83f6-6a8258908351 admin admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:14:33.038944 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c033bf8-a87f-4794-83f6-6a8258908351 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:14:33.039214 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3c033bf8-a87f-4794-83f6-6a8258908351 admin admin] PATH_INFO: `/v3/roles` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:14:33.039944 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c033bf8-a87f-4794-83f6-6a8258908351 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:14:33.041678 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3c033bf8-a87f-4794-83f6-6a8258908351 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:14:33.051319 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1066/2138] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jul 13 22:14:32 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 22:14:33.055699 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dfc2724-71a6-48a8-8913-668ec14d8fd3 admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:14:33.061852 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:14:33.068590 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7dfc2724-71a6-48a8-8913-668ec14d8fd3 admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:14:33.075503 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa7dbe8-fabd-422a-8e10-2fadef38c1e4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:14:33.076440 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa7dbe8-fabd-422a-8e10-2fadef38c1e4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:14:33.076777 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa7dbe8-fabd-422a-8e10-2fadef38c1e4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:14:33.077051 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffa7dbe8-fabd-422a-8e10-2fadef38c1e4 admin admin] PATH_INFO: `/v3/projects/6090275139944ca28ea6fcf64b8b5aeb/users/7834297c19ec4593a9626d5fecd60fd0/roles/44efba2579f342df8a0789c59d163ec3` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:14:33.079346 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa7dbe8-fabd-422a-8e10-2fadef38c1e4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6090275139944ca28ea6fcf64b8b5aeb, user_id=7834297c19ec4593a9626d5fecd60fd0, role_id=44efba2579f342df8a0789c59d163ec3)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:14:33.082286 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ffa7dbe8-fabd-422a-8e10-2fadef38c1e4 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:14:33.093340 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1073/2139] 10.222.0.52 () {60 vars in 1482 bytes} [Wed Jul 13 22:14:33 2022] PUT /identity/v3/projects/6090275139944ca28ea6fcf64b8b5aeb/users/7834297c19ec4593a9626d5fecd60fd0/roles/44efba2579f342df8a0789c59d163ec3 => generated 0 bytes in 38 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 13 22:14:33.103866 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5a95e4-2ee4-40ff-bfc8-8054326cf4b4 None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:14:33.104158 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5a95e4-2ee4-40ff-bfc8-8054326cf4b4 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:14:33.104429 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3a5a95e4-2ee4-40ff-bfc8-8054326cf4b4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:14:33.132004 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-3a5a95e4-2ee4-40ff-bfc8-8054326cf4b4 None None] MFA Rules not processed for user `7834297c19ec4593a9626d5fecd60fd0`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 22:14:33.133252 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-3a5a95e4-2ee4-40ff-bfc8-8054326cf4b4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:14:33.208080 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1067/2140] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jul 13 22:14:33 2022] POST /identity/v3/auth/tokens => generated 3333 bytes in 105 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 22:14:33.220863 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa7dbe8-fabd-422a-8e10-2fadef38c1e4 admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:14:33.222988 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-ffa7dbe8-fabd-422a-8e10-2fadef38c1e4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:14:33.284838 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffa7dbe8-fabd-422a-8e10-2fadef38c1e4 admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:14:33.288881 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8946dca6-deb2-4992-9982-6f39cf91b88a service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc77e1ea1ada4700b809beb1c980c5e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417e3e2f1333481795a089bef2e24bea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:14:33.289514 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8946dca6-deb2-4992-9982-6f39cf91b88a service cinder] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:14:33.289786 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8946dca6-deb2-4992-9982-6f39cf91b88a service cinder] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:14:33.290054 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8946dca6-deb2-4992-9982-6f39cf91b88a service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:14:33.299613 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8946dca6-deb2-4992-9982-6f39cf91b88a service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:14:33.301047 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8946dca6-deb2-4992-9982-6f39cf91b88a service cinder] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:14:33.311230 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1074/2141] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jul 13 22:14:33 2022] GET /identity/v3/auth/tokens => generated 3333 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 22:14:33.402598 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5a95e4-2ee4-40ff-bfc8-8054326cf4b4 None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:14:33.412863 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3a5a95e4-2ee4-40ff-bfc8-8054326cf4b4 None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:14:33.417588 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d7c466f0-1bb2-4176-9fef-cd9ec78e6ba3 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc77e1ea1ada4700b809beb1c980c5e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417e3e2f1333481795a089bef2e24bea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:14:33.418320 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c466f0-1bb2-4176-9fef-cd9ec78e6ba3 service cinder] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:14:33.418647 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c466f0-1bb2-4176-9fef-cd9ec78e6ba3 service cinder] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:14:33.419223 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d7c466f0-1bb2-4176-9fef-cd9ec78e6ba3 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:14:33.483795 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c466f0-1bb2-4176-9fef-cd9ec78e6ba3 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:14:33.485151 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d7c466f0-1bb2-4176-9fef-cd9ec78e6ba3 service cinder] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:14:33.494226 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1068/2142] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jul 13 22:14:33 2022] GET /identity/v3/auth/tokens => generated 3268 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 22:14:37.650722 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8946dca6-deb2-4992-9982-6f39cf91b88a service cinder] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:14:37.718562 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8946dca6-deb2-4992-9982-6f39cf91b88a service cinder] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:14:37.723540 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d16f5f-53a8-489e-8712-fa63edc5f817 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:14:37.724444 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d16f5f-53a8-489e-8712-fa63edc5f817 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:14:37.724776 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d16f5f-53a8-489e-8712-fa63edc5f817 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:14:37.725100 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-76d16f5f-53a8-489e-8712-fa63edc5f817 admin admin] PATH_INFO: `/v3/users/5d5cd89b7edb4eadbdfec0dba1e2dee8` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:14:37.726433 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d16f5f-53a8-489e-8712-fa63edc5f817 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5d5cd89b7edb4eadbdfec0dba1e2dee8)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:14:37.728598 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-76d16f5f-53a8-489e-8712-fa63edc5f817 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:14:37.783492 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-76d16f5f-53a8-489e-8712-fa63edc5f817 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5d5cd89b7edb4eadbdfec0dba1e2dee8'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:14:37.790808 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-76d16f5f-53a8-489e-8712-fa63edc5f817 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5d5cd89b7edb4eadbdfec0dba1e2dee8'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:14:37.810280 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-76d16f5f-53a8-489e-8712-fa63edc5f817 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5d5cd89b7edb4eadbdfec0dba1e2dee8'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:14:37.811307 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-76d16f5f-53a8-489e-8712-fa63edc5f817 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5d5cd89b7edb4eadbdfec0dba1e2dee8'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:14:37.812004 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-76d16f5f-53a8-489e-8712-fa63edc5f817 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5d5cd89b7edb4eadbdfec0dba1e2dee8'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:14:37.825653 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1075/2143] 10.222.0.52 () {58 vars in 1223 bytes} [Wed Jul 13 22:14:37 2022] DELETE /identity/v3/users/5d5cd89b7edb4eadbdfec0dba1e2dee8 => generated 0 bytes in 175 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 22:14:37.830250 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9734eb-301c-44c8-8525-22cfa4487961 None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:14:37.830793 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9734eb-301c-44c8-8525-22cfa4487961 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:14:37.830881 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd9734eb-301c-44c8-8525-22cfa4487961 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:14:37.848644 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-fd9734eb-301c-44c8-8525-22cfa4487961 None None] MFA Rules not processed for user `cc99c93da5074cf5a48bc2aa9a92c5e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 22:14:37.849850 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-fd9734eb-301c-44c8-8525-22cfa4487961 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:14:37.897828 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:14:37.917785 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1069/2144] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jul 13 22:14:37 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 22:14:37.927699 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d16f5f-53a8-489e-8712-fa63edc5f817 admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:14:37.929404 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-76d16f5f-53a8-489e-8712-fa63edc5f817 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:14:37.943740 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:14:37.989156 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-76d16f5f-53a8-489e-8712-fa63edc5f817 admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:14:37.993936 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6925b0c-dd63-460f-9542-997ca992b376 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '847ec9a1bd4d4dec804318a5c845895a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417e3e2f1333481795a089bef2e24bea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:14:37.994897 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6925b0c-dd63-460f-9542-997ca992b376 service neutron] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:14:37.994897 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6925b0c-dd63-460f-9542-997ca992b376 service neutron] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:14:37.995164 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6925b0c-dd63-460f-9542-997ca992b376 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:14:38.005605 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6925b0c-dd63-460f-9542-997ca992b376 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:14:38.007097 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6925b0c-dd63-460f-9542-997ca992b376 service neutron] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:14:38.018525 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1076/2145] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jul 13 22:14:37 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 22:14:38.337967 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9734eb-301c-44c8-8525-22cfa4487961 None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:14:38.339502 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-fd9734eb-301c-44c8-8525-22cfa4487961 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:14:38.357657 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd9734eb-301c-44c8-8525-22cfa4487961 None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:14:38.363036 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69db8a96-41b2-49ba-abf5-c73404ab89b0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:14:38.365080 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69db8a96-41b2-49ba-abf5-c73404ab89b0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:14:38.365435 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69db8a96-41b2-49ba-abf5-c73404ab89b0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:14:38.365804 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69db8a96-41b2-49ba-abf5-c73404ab89b0 admin admin] PATH_INFO: `/v3/projects/ed432eceba0645e8a1039f65b1a45e8d` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:14:38.367229 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69db8a96-41b2-49ba-abf5-c73404ab89b0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ed432eceba0645e8a1039f65b1a45e8d)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:14:38.369681 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-69db8a96-41b2-49ba-abf5-c73404ab89b0 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:14:38.424803 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-69db8a96-41b2-49ba-abf5-c73404ab89b0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ed432eceba0645e8a1039f65b1a45e8d', 'request_id': 'req-69db8a96-41b2-49ba-abf5-c73404ab89b0'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:14:38.431136 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-69db8a96-41b2-49ba-abf5-c73404ab89b0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ed432eceba0645e8a1039f65b1a45e8d', 'request_id': 'req-69db8a96-41b2-49ba-abf5-c73404ab89b0'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:14:38.436887 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-69db8a96-41b2-49ba-abf5-c73404ab89b0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ed432eceba0645e8a1039f65b1a45e8d', 'request_id': 'req-69db8a96-41b2-49ba-abf5-c73404ab89b0'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:14:38.454462 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-69db8a96-41b2-49ba-abf5-c73404ab89b0 admin admin] The token cache is being invalidate because project ed432eceba0645e8a1039f65b1a45e8d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80382) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 22:14:38.454878 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-69db8a96-41b2-49ba-abf5-c73404ab89b0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:14:38.455416 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-69db8a96-41b2-49ba-abf5-c73404ab89b0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:14:38.457013 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1070/2146] 10.222.0.52 () {58 vars in 1232 bytes} [Wed Jul 13 22:14:38 2022] DELETE /identity/v3/projects/ed432eceba0645e8a1039f65b1a45e8d => generated 0 bytes in 120 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 13 22:14:38.461083 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6925b0c-dd63-460f-9542-997ca992b376 service neutron] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:14:38.471573 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6925b0c-dd63-460f-9542-997ca992b376 service neutron] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:14:38.533176 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-213d5acf-074b-40af-864a-098b962a09c4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:14:38.534356 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213d5acf-074b-40af-864a-098b962a09c4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:14:38.535077 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213d5acf-074b-40af-864a-098b962a09c4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:14:38.535352 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-213d5acf-074b-40af-864a-098b962a09c4 admin admin] PATH_INFO: `/v3/users/7834297c19ec4593a9626d5fecd60fd0` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:14:38.536989 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213d5acf-074b-40af-864a-098b962a09c4 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7834297c19ec4593a9626d5fecd60fd0)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:14:38.540071 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-213d5acf-074b-40af-864a-098b962a09c4 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:14:38.594814 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-213d5acf-074b-40af-864a-098b962a09c4 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7834297c19ec4593a9626d5fecd60fd0'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:14:38.603974 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-213d5acf-074b-40af-864a-098b962a09c4 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7834297c19ec4593a9626d5fecd60fd0'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:14:38.616075 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-213d5acf-074b-40af-864a-098b962a09c4 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7834297c19ec4593a9626d5fecd60fd0'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:14:38.616731 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-213d5acf-074b-40af-864a-098b962a09c4 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7834297c19ec4593a9626d5fecd60fd0'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:14:38.617328 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-213d5acf-074b-40af-864a-098b962a09c4 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7834297c19ec4593a9626d5fecd60fd0'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:14:38.630723 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1077/2147] 10.222.0.52 () {58 vars in 1223 bytes} [Wed Jul 13 22:14:38 2022] DELETE /identity/v3/users/7834297c19ec4593a9626d5fecd60fd0 => generated 0 bytes in 170 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 22:14:38.919841 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69db8a96-41b2-49ba-abf5-c73404ab89b0 admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:14:38.921474 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-69db8a96-41b2-49ba-abf5-c73404ab89b0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:14:38.938717 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-69db8a96-41b2-49ba-abf5-c73404ab89b0 admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:14:39.004107 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-45aabce5-9cff-407a-8000-17e49094f9fb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:14:39.005010 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45aabce5-9cff-407a-8000-17e49094f9fb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:14:39.005358 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45aabce5-9cff-407a-8000-17e49094f9fb admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:14:39.005656 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-45aabce5-9cff-407a-8000-17e49094f9fb admin admin] PATH_INFO: `/v3/projects/6090275139944ca28ea6fcf64b8b5aeb` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:14:39.007047 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45aabce5-9cff-407a-8000-17e49094f9fb admin admin] RBAC: Authorizing `identity:delete_project(project_id=6090275139944ca28ea6fcf64b8b5aeb)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:14:39.009066 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-45aabce5-9cff-407a-8000-17e49094f9fb admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:14:39.058922 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-45aabce5-9cff-407a-8000-17e49094f9fb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6090275139944ca28ea6fcf64b8b5aeb', 'request_id': 'req-45aabce5-9cff-407a-8000-17e49094f9fb'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:14:39.065589 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-45aabce5-9cff-407a-8000-17e49094f9fb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6090275139944ca28ea6fcf64b8b5aeb', 'request_id': 'req-45aabce5-9cff-407a-8000-17e49094f9fb'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:14:39.073956 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-45aabce5-9cff-407a-8000-17e49094f9fb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6090275139944ca28ea6fcf64b8b5aeb', 'request_id': 'req-45aabce5-9cff-407a-8000-17e49094f9fb'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:14:39.087037 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=84,keystone:DELETE=14,keystone:INSERT=2 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:14:39.092446 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-45aabce5-9cff-407a-8000-17e49094f9fb admin admin] The token cache is being invalidate because project 6090275139944ca28ea6fcf64b8b5aeb was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80382) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 22:14:39.092802 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-45aabce5-9cff-407a-8000-17e49094f9fb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:14:39.093498 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-45aabce5-9cff-407a-8000-17e49094f9fb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:14:39.095499 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1071/2148] 10.222.0.52 () {58 vars in 1232 bytes} [Wed Jul 13 22:14:38 2022] DELETE /identity/v3/projects/6090275139944ca28ea6fcf64b8b5aeb => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 22:14:39.100351 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351adb7a-ebfe-4414-9ef9-69f7fb0c95c7 None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:14:39.100840 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351adb7a-ebfe-4414-9ef9-69f7fb0c95c7 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:14:39.100943 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-351adb7a-ebfe-4414-9ef9-69f7fb0c95c7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:14:39.120848 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-351adb7a-ebfe-4414-9ef9-69f7fb0c95c7 None None] MFA Rules not processed for user `cc99c93da5074cf5a48bc2aa9a92c5e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 22:14:39.122100 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-351adb7a-ebfe-4414-9ef9-69f7fb0c95c7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:14:39.152623 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:14:39.187995 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1078/2149] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jul 13 22:14:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 89 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 22:14:39.197500 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0942d918-0b42-4097-a67f-ef5b3967d08b None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:14:39.197752 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0942d918-0b42-4097-a67f-ef5b3967d08b None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:14:39.198044 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0942d918-0b42-4097-a67f-ef5b3967d08b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:14:39.217679 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-0942d918-0b42-4097-a67f-ef5b3967d08b None None] MFA Rules not processed for user `cc99c93da5074cf5a48bc2aa9a92c5e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 22:14:39.218927 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-0942d918-0b42-4097-a67f-ef5b3967d08b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:14:39.227395 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1072/2150] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jul 13 22:14:39 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 22:14:39.231709 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351adb7a-ebfe-4414-9ef9-69f7fb0c95c7 None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:14:39.242171 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-351adb7a-ebfe-4414-9ef9-69f7fb0c95c7 None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:14:39.247021 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b75c9ea-d49e-4f8e-b3c3-5357aab7b2d5 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:14:39.247683 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b75c9ea-d49e-4f8e-b3c3-5357aab7b2d5 admin admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:14:39.247957 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b75c9ea-d49e-4f8e-b3c3-5357aab7b2d5 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:14:39.248222 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b75c9ea-d49e-4f8e-b3c3-5357aab7b2d5 admin admin] PATH_INFO: `/v3/domains` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:14:39.248986 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b75c9ea-d49e-4f8e-b3c3-5357aab7b2d5 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 22:14:39.249288 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b75c9ea-d49e-4f8e-b3c3-5357aab7b2d5 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:14:39.250928 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b75c9ea-d49e-4f8e-b3c3-5357aab7b2d5 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:14:39.263199 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1079/2151] 10.222.0.52 () {58 vars in 1152 bytes} [Wed Jul 13 22:14:39 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 22:14:39.267664 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0942d918-0b42-4097-a67f-ef5b3967d08b None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:14:39.276993 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0942d918-0b42-4097-a67f-ef5b3967d08b None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:14:39.282583 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1fe44b-d4e2-4642-a1d3-bd62de2e8e60 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:14:39.283365 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1fe44b-d4e2-4642-a1d3-bd62de2e8e60 admin admin] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:14:39.283713 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1fe44b-d4e2-4642-a1d3-bd62de2e8e60 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:14:39.283996 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a1fe44b-d4e2-4642-a1d3-bd62de2e8e60 admin admin] PATH_INFO: `/v3/projects` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:14:39.284725 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a1fe44b-d4e2-4642-a1d3-bd62de2e8e60 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:14:39.286754 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a1fe44b-d4e2-4642-a1d3-bd62de2e8e60 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:14:39.319089 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1073/2152] 10.222.0.52 () {60 vars in 1152 bytes} [Wed Jul 13 22:14:39 2022] POST /identity/v3/projects => generated 387 bytes in 52 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 22:14:39.323669 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b75c9ea-d49e-4f8e-b3c3-5357aab7b2d5 admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:14:39.334653 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b75c9ea-d49e-4f8e-b3c3-5357aab7b2d5 admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:14:39.394131 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a082603f-f3d6-4771-8e73-d4010f56bcae admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:14:39.394845 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a082603f-f3d6-4771-8e73-d4010f56bcae admin admin] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:14:39.395129 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a082603f-f3d6-4771-8e73-d4010f56bcae admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:14:39.395405 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a082603f-f3d6-4771-8e73-d4010f56bcae admin admin] PATH_INFO: `/v3/users` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:14:39.396101 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a082603f-f3d6-4771-8e73-d4010f56bcae admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:14:39.398455 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a082603f-f3d6-4771-8e73-d4010f56bcae admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:14:39.420091 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1080/2153] 10.222.0.52 () {60 vars in 1143 bytes} [Wed Jul 13 22:14:39 2022] POST /identity/v3/users => generated 294 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 22:14:39.424929 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1fe44b-d4e2-4642-a1d3-bd62de2e8e60 admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:14:39.436533 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a1fe44b-d4e2-4642-a1d3-bd62de2e8e60 admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:14:39.442119 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a962a33b-b833-412b-9313-72e6a1b9d938 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:14:39.442636 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a962a33b-b833-412b-9313-72e6a1b9d938 admin admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:14:39.442800 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a962a33b-b833-412b-9313-72e6a1b9d938 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:14:39.443079 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a962a33b-b833-412b-9313-72e6a1b9d938 admin admin] PATH_INFO: `/v3/roles` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:14:39.443821 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a962a33b-b833-412b-9313-72e6a1b9d938 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:14:39.445453 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a962a33b-b833-412b-9313-72e6a1b9d938 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:14:39.456994 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1074/2154] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jul 13 22:14:39 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 22:14:39.461115 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a082603f-f3d6-4771-8e73-d4010f56bcae admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:14:39.475353 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a082603f-f3d6-4771-8e73-d4010f56bcae admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:14:39.480431 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c0edaef-a541-4cb7-8520-f3501b02bcdd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:14:39.481169 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0edaef-a541-4cb7-8520-f3501b02bcdd admin admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:14:39.481456 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0edaef-a541-4cb7-8520-f3501b02bcdd admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:14:39.481731 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c0edaef-a541-4cb7-8520-f3501b02bcdd admin admin] PATH_INFO: `/v3/roles` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:14:39.482468 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0edaef-a541-4cb7-8520-f3501b02bcdd admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:14:39.484121 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c0edaef-a541-4cb7-8520-f3501b02bcdd admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:14:39.494815 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1081/2155] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jul 13 22:14:39 2022] GET /identity/v3/roles => generated 1420 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 22:14:39.499095 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a962a33b-b833-412b-9313-72e6a1b9d938 admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:14:39.509721 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a962a33b-b833-412b-9313-72e6a1b9d938 admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:14:39.514667 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e486b4-99a0-4364-82e9-4bfa8bcdd3ce admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:14:39.515576 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e486b4-99a0-4364-82e9-4bfa8bcdd3ce admin admin] REQUEST_METHOD: `PUT` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:14:39.515854 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e486b4-99a0-4364-82e9-4bfa8bcdd3ce admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:14:39.516126 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79e486b4-99a0-4364-82e9-4bfa8bcdd3ce admin admin] PATH_INFO: `/v3/projects/e3442984990b404c88c03b0c5d1d89aa/users/1fc9535825ab4eccbcebf2842aa72b04/roles/2a5e86976394498abb6310697cdb3f8b` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:14:39.532032 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e486b4-99a0-4364-82e9-4bfa8bcdd3ce admin admin] RBAC: Authorizing `identity:create_grant(project_id=e3442984990b404c88c03b0c5d1d89aa, user_id=1fc9535825ab4eccbcebf2842aa72b04, role_id=2a5e86976394498abb6310697cdb3f8b)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:14:39.534808 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79e486b4-99a0-4364-82e9-4bfa8bcdd3ce admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:14:39.559983 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1075/2156] 10.222.0.52 () {60 vars in 1482 bytes} [Wed Jul 13 22:14:39 2022] PUT /identity/v3/projects/e3442984990b404c88c03b0c5d1d89aa/users/1fc9535825ab4eccbcebf2842aa72b04/roles/2a5e86976394498abb6310697cdb3f8b => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 22:14:39.572631 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706220eb-717c-44c2-b220-b0b873c0daf1 None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:14:39.572863 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706220eb-717c-44c2-b220-b0b873c0daf1 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:14:39.573234 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-706220eb-717c-44c2-b220-b0b873c0daf1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:14:39.601831 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-706220eb-717c-44c2-b220-b0b873c0daf1 None None] MFA Rules not processed for user `1fc9535825ab4eccbcebf2842aa72b04`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 22:14:39.603358 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-706220eb-717c-44c2-b220-b0b873c0daf1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:14:39.677341 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1082/2157] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jul 13 22:14:39 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 22:14:39.681984 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e486b4-99a0-4364-82e9-4bfa8bcdd3ce admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:14:39.750984 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79e486b4-99a0-4364-82e9-4bfa8bcdd3ce admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:14:39.756142 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed56e756-f7b3-452f-8cc8-6664a4e1a426 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:14:39.756801 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed56e756-f7b3-452f-8cc8-6664a4e1a426 admin admin] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:14:39.757079 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed56e756-f7b3-452f-8cc8-6664a4e1a426 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:14:39.757353 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed56e756-f7b3-452f-8cc8-6664a4e1a426 admin admin] PATH_INFO: `/v3/projects` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:14:39.758065 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed56e756-f7b3-452f-8cc8-6664a4e1a426 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:14:39.760025 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed56e756-f7b3-452f-8cc8-6664a4e1a426 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:14:39.789303 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1076/2158] 10.222.0.52 () {60 vars in 1152 bytes} [Wed Jul 13 22:14:39 2022] POST /identity/v3/projects => generated 389 bytes in 108 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 22:14:39.794197 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706220eb-717c-44c2-b220-b0b873c0daf1 None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:14:39.806055 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-706220eb-717c-44c2-b220-b0b873c0daf1 None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:14:39.833384 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:14:39.866975 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6227be4-a868-44c8-be84-48e51baf7137 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:14:39.867654 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6227be4-a868-44c8-be84-48e51baf7137 admin admin] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:14:39.867938 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6227be4-a868-44c8-be84-48e51baf7137 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:14:39.868212 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6227be4-a868-44c8-be84-48e51baf7137 admin admin] PATH_INFO: `/v3/users` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:14:39.868908 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6227be4-a868-44c8-be84-48e51baf7137 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:14:39.870948 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6227be4-a868-44c8-be84-48e51baf7137 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:14:39.893880 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1083/2159] 10.222.0.52 () {60 vars in 1143 bytes} [Wed Jul 13 22:14:39 2022] POST /identity/v3/users => generated 301 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 22:14:39.898954 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed56e756-f7b3-452f-8cc8-6664a4e1a426 admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:14:39.912483 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed56e756-f7b3-452f-8cc8-6664a4e1a426 admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:14:39.919627 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ace2bad-5ae2-4ba5-af8f-f3a9f43987e6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:14:39.920327 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ace2bad-5ae2-4ba5-af8f-f3a9f43987e6 admin admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:14:39.920877 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ace2bad-5ae2-4ba5-af8f-f3a9f43987e6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:14:39.920877 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2ace2bad-5ae2-4ba5-af8f-f3a9f43987e6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:14:39.921633 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ace2bad-5ae2-4ba5-af8f-f3a9f43987e6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:14:39.923404 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2ace2bad-5ae2-4ba5-af8f-f3a9f43987e6 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:14:39.935047 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1077/2160] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jul 13 22:14:39 2022] GET /identity/v3/roles => generated 1420 bytes in 37 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 22:14:39.939529 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6227be4-a868-44c8-be84-48e51baf7137 admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:14:39.950160 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6227be4-a868-44c8-be84-48e51baf7137 admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:14:39.955987 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9c1eb0-d691-4d82-bb60-bd80815df129 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:14:39.956907 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9c1eb0-d691-4d82-bb60-bd80815df129 admin admin] REQUEST_METHOD: `PUT` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:14:39.957183 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9c1eb0-d691-4d82-bb60-bd80815df129 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:14:39.957445 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0b9c1eb0-d691-4d82-bb60-bd80815df129 admin admin] PATH_INFO: `/v3/projects/ac770309087d4370844e15a4b62ae22b/users/7d4a63c3d7af45dcbd7edb4989811a9a/roles/44efba2579f342df8a0789c59d163ec3` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:14:39.973745 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9c1eb0-d691-4d82-bb60-bd80815df129 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ac770309087d4370844e15a4b62ae22b, user_id=7d4a63c3d7af45dcbd7edb4989811a9a, role_id=44efba2579f342df8a0789c59d163ec3)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:14:39.976436 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0b9c1eb0-d691-4d82-bb60-bd80815df129 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:14:39.992240 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1084/2161] 10.222.0.52 () {60 vars in 1482 bytes} [Wed Jul 13 22:14:39 2022] PUT /identity/v3/projects/ac770309087d4370844e15a4b62ae22b/users/7d4a63c3d7af45dcbd7edb4989811a9a/roles/44efba2579f342df8a0789c59d163ec3 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 13 22:14:39.996573 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ace2bad-5ae2-4ba5-af8f-f3a9f43987e6 admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:14:40.007349 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2ace2bad-5ae2-4ba5-af8f-f3a9f43987e6 admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:14:40.071493 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98be990-92e0-4b37-bfbe-bd07289e9ccb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:14:40.072729 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98be990-92e0-4b37-bfbe-bd07289e9ccb admin admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:14:40.073535 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98be990-92e0-4b37-bfbe-bd07289e9ccb admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:14:40.073916 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98be990-92e0-4b37-bfbe-bd07289e9ccb admin admin] PATH_INFO: `/v3/roles` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:14:40.075065 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e98be990-92e0-4b37-bfbe-bd07289e9ccb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:14:40.077397 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e98be990-92e0-4b37-bfbe-bd07289e9ccb admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:14:40.088516 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1078/2162] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jul 13 22:14:39 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 22:14:40.093016 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9c1eb0-d691-4d82-bb60-bd80815df129 admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:14:40.104652 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0b9c1eb0-d691-4d82-bb60-bd80815df129 admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:14:40.110709 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebbb37e-fec5-4c8c-a240-7182bdc0b397 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:14:40.111967 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebbb37e-fec5-4c8c-a240-7182bdc0b397 admin admin] REQUEST_METHOD: `PUT` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:14:40.112249 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebbb37e-fec5-4c8c-a240-7182bdc0b397 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:14:40.112523 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ebbb37e-fec5-4c8c-a240-7182bdc0b397 admin admin] PATH_INFO: `/v3/projects/ac770309087d4370844e15a4b62ae22b/users/7d4a63c3d7af45dcbd7edb4989811a9a/roles/44efba2579f342df8a0789c59d163ec3` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:14:40.114691 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ebbb37e-fec5-4c8c-a240-7182bdc0b397 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ac770309087d4370844e15a4b62ae22b, user_id=7d4a63c3d7af45dcbd7edb4989811a9a, role_id=44efba2579f342df8a0789c59d163ec3)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:14:40.117333 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ebbb37e-fec5-4c8c-a240-7182bdc0b397 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:14:40.128291 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1085/2163] 10.222.0.52 () {60 vars in 1482 bytes} [Wed Jul 13 22:14:40 2022] PUT /identity/v3/projects/ac770309087d4370844e15a4b62ae22b/users/7d4a63c3d7af45dcbd7edb4989811a9a/roles/44efba2579f342df8a0789c59d163ec3 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 22:14:40.138561 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d646132-259d-498e-a061-0ff19a67196b None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:14:40.138872 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d646132-259d-498e-a061-0ff19a67196b None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:14:40.139140 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7d646132-259d-498e-a061-0ff19a67196b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:14:40.146560 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:14:40.167887 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-7d646132-259d-498e-a061-0ff19a67196b None None] MFA Rules not processed for user `7d4a63c3d7af45dcbd7edb4989811a9a`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 22:14:40.169126 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-7d646132-259d-498e-a061-0ff19a67196b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:14:40.245760 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1079/2164] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jul 13 22:14:40 2022] POST /identity/v3/auth/tokens => generated 3323 bytes in 108 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 22:14:40.257580 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebbb37e-fec5-4c8c-a240-7182bdc0b397 admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:14:40.259634 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-8ebbb37e-fec5-4c8c-a240-7182bdc0b397 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:14:40.320903 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ebbb37e-fec5-4c8c-a240-7182bdc0b397 admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:14:40.325428 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f5969a88-8f01-4e37-81dc-d1350a2d4f96 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc77e1ea1ada4700b809beb1c980c5e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417e3e2f1333481795a089bef2e24bea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:14:40.326011 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5969a88-8f01-4e37-81dc-d1350a2d4f96 service cinder] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:14:40.326281 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5969a88-8f01-4e37-81dc-d1350a2d4f96 service cinder] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:14:40.326583 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f5969a88-8f01-4e37-81dc-d1350a2d4f96 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:14:40.336103 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5969a88-8f01-4e37-81dc-d1350a2d4f96 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:14:40.337552 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f5969a88-8f01-4e37-81dc-d1350a2d4f96 service cinder] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:14:40.348653 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1086/2165] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jul 13 22:14:40 2022] GET /identity/v3/auth/tokens => generated 3323 bytes in 92 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 22:14:40.639893 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d646132-259d-498e-a061-0ff19a67196b None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:14:40.705527 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7d646132-259d-498e-a061-0ff19a67196b None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:14:40.710731 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18971269-2215-4ffe-a890-a6851023038b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:14:40.711611 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18971269-2215-4ffe-a890-a6851023038b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:14:40.711882 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18971269-2215-4ffe-a890-a6851023038b admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:14:40.712165 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18971269-2215-4ffe-a890-a6851023038b admin admin] PATH_INFO: `/v3/users/1fc9535825ab4eccbcebf2842aa72b04` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:14:40.713385 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18971269-2215-4ffe-a890-a6851023038b admin admin] RBAC: Authorizing `identity:delete_user(user_id=1fc9535825ab4eccbcebf2842aa72b04)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:14:40.715382 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18971269-2215-4ffe-a890-a6851023038b admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:14:40.763381 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-18971269-2215-4ffe-a890-a6851023038b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '1fc9535825ab4eccbcebf2842aa72b04'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:14:40.772901 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-18971269-2215-4ffe-a890-a6851023038b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '1fc9535825ab4eccbcebf2842aa72b04'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:14:40.785134 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-18971269-2215-4ffe-a890-a6851023038b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '1fc9535825ab4eccbcebf2842aa72b04'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:14:40.785794 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-18971269-2215-4ffe-a890-a6851023038b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '1fc9535825ab4eccbcebf2842aa72b04'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:14:40.786443 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-18971269-2215-4ffe-a890-a6851023038b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '1fc9535825ab4eccbcebf2842aa72b04'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:14:40.799288 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1080/2166] 10.222.0.52 () {58 vars in 1223 bytes} [Wed Jul 13 22:14:40 2022] DELETE /identity/v3/users/1fc9535825ab4eccbcebf2842aa72b04 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 22:14:40.804928 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92dced2a-9a33-41ba-b1ff-a83e77fb800b None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:14:40.805334 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92dced2a-9a33-41ba-b1ff-a83e77fb800b None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:14:40.805673 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92dced2a-9a33-41ba-b1ff-a83e77fb800b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:14:40.827746 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-92dced2a-9a33-41ba-b1ff-a83e77fb800b None None] MFA Rules not processed for user `cc99c93da5074cf5a48bc2aa9a92c5e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 22:14:40.828872 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-92dced2a-9a33-41ba-b1ff-a83e77fb800b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:14:40.891451 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1087/2167] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jul 13 22:14:40 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 22:14:40.901200 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18971269-2215-4ffe-a890-a6851023038b admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:14:40.902760 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-18971269-2215-4ffe-a890-a6851023038b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:14:40.913727 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:14:40.963693 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18971269-2215-4ffe-a890-a6851023038b admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:14:40.968630 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802e7b15-c7c0-4e4d-bacf-f4f2562cd83c service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '847ec9a1bd4d4dec804318a5c845895a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417e3e2f1333481795a089bef2e24bea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:14:40.969392 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802e7b15-c7c0-4e4d-bacf-f4f2562cd83c service neutron] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:14:40.969649 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802e7b15-c7c0-4e4d-bacf-f4f2562cd83c service neutron] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:14:40.970008 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-802e7b15-c7c0-4e4d-bacf-f4f2562cd83c service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:14:40.983107 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-802e7b15-c7c0-4e4d-bacf-f4f2562cd83c service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:14:40.985976 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-802e7b15-c7c0-4e4d-bacf-f4f2562cd83c service neutron] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:14:41.009724 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1081/2168] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jul 13 22:14:40 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 109 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 22:14:41.280811 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92dced2a-9a33-41ba-b1ff-a83e77fb800b None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:14:41.282452 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-92dced2a-9a33-41ba-b1ff-a83e77fb800b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:14:41.295037 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-92dced2a-9a33-41ba-b1ff-a83e77fb800b None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:14:41.300707 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ad0ccf-6888-4ed2-965c-b1732d0ff770 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:14:41.302692 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ad0ccf-6888-4ed2-965c-b1732d0ff770 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:14:41.303348 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ad0ccf-6888-4ed2-965c-b1732d0ff770 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:14:41.303676 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2ad0ccf-6888-4ed2-965c-b1732d0ff770 admin admin] PATH_INFO: `/v3/projects/e3442984990b404c88c03b0c5d1d89aa` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:14:41.304833 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ad0ccf-6888-4ed2-965c-b1732d0ff770 admin admin] RBAC: Authorizing `identity:delete_project(project_id=e3442984990b404c88c03b0c5d1d89aa)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:14:41.307007 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2ad0ccf-6888-4ed2-965c-b1732d0ff770 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:14:41.331456 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:14:41.363049 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-a2ad0ccf-6888-4ed2-965c-b1732d0ff770 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'e3442984990b404c88c03b0c5d1d89aa', 'request_id': 'req-a2ad0ccf-6888-4ed2-965c-b1732d0ff770'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:14:41.372243 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-a2ad0ccf-6888-4ed2-965c-b1732d0ff770 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'e3442984990b404c88c03b0c5d1d89aa', 'request_id': 'req-a2ad0ccf-6888-4ed2-965c-b1732d0ff770'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:14:41.379774 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-a2ad0ccf-6888-4ed2-965c-b1732d0ff770 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'e3442984990b404c88c03b0c5d1d89aa', 'request_id': 'req-a2ad0ccf-6888-4ed2-965c-b1732d0ff770'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:14:41.396063 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-a2ad0ccf-6888-4ed2-965c-b1732d0ff770 admin admin] The token cache is being invalidate because project e3442984990b404c88c03b0c5d1d89aa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80381) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 22:14:41.396346 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-a2ad0ccf-6888-4ed2-965c-b1732d0ff770 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:14:41.397007 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-a2ad0ccf-6888-4ed2-965c-b1732d0ff770 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:14:41.398611 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1088/2169] 10.222.0.52 () {58 vars in 1232 bytes} [Wed Jul 13 22:14:41 2022] DELETE /identity/v3/projects/e3442984990b404c88c03b0c5d1d89aa => generated 0 bytes in 118 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 22:14:41.403979 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802e7b15-c7c0-4e4d-bacf-f4f2562cd83c service neutron] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:14:41.414175 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-802e7b15-c7c0-4e4d-bacf-f4f2562cd83c service neutron] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:14:41.477258 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-33291677-51d0-4b16-b107-7b0be11bafda admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:14:41.478163 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33291677-51d0-4b16-b107-7b0be11bafda admin admin] REQUEST_METHOD: `DELETE` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:14:41.478445 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33291677-51d0-4b16-b107-7b0be11bafda admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:14:41.478757 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-33291677-51d0-4b16-b107-7b0be11bafda admin admin] PATH_INFO: `/v3/users/7d4a63c3d7af45dcbd7edb4989811a9a` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:14:41.479987 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33291677-51d0-4b16-b107-7b0be11bafda admin admin] RBAC: Authorizing `identity:delete_user(user_id=7d4a63c3d7af45dcbd7edb4989811a9a)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:14:41.481928 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-33291677-51d0-4b16-b107-7b0be11bafda admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:14:41.538915 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-33291677-51d0-4b16-b107-7b0be11bafda admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7d4a63c3d7af45dcbd7edb4989811a9a'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:14:41.548392 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-33291677-51d0-4b16-b107-7b0be11bafda admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7d4a63c3d7af45dcbd7edb4989811a9a'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:14:41.567528 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-33291677-51d0-4b16-b107-7b0be11bafda admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7d4a63c3d7af45dcbd7edb4989811a9a'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:14:41.568354 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-33291677-51d0-4b16-b107-7b0be11bafda admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7d4a63c3d7af45dcbd7edb4989811a9a'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:14:41.569006 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-33291677-51d0-4b16-b107-7b0be11bafda admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7d4a63c3d7af45dcbd7edb4989811a9a'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:14:41.585333 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1082/2170] 10.222.0.52 () {58 vars in 1223 bytes} [Wed Jul 13 22:14:41 2022] DELETE /identity/v3/users/7d4a63c3d7af45dcbd7edb4989811a9a => generated 0 bytes in 182 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 22:14:41.927823 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ad0ccf-6888-4ed2-965c-b1732d0ff770 admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:14:41.929360 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-a2ad0ccf-6888-4ed2-965c-b1732d0ff770 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:14:41.943976 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2ad0ccf-6888-4ed2-965c-b1732d0ff770 admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:14:42.025286 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3bad1e0-a9c4-4085-8986-24d17e7a5bab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:14:42.025905 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bad1e0-a9c4-4085-8986-24d17e7a5bab admin admin] REQUEST_METHOD: `DELETE` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:14:42.026187 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bad1e0-a9c4-4085-8986-24d17e7a5bab admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:14:42.026457 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3bad1e0-a9c4-4085-8986-24d17e7a5bab admin admin] PATH_INFO: `/v3/projects/ac770309087d4370844e15a4b62ae22b` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:14:42.027890 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3bad1e0-a9c4-4085-8986-24d17e7a5bab admin admin] RBAC: Authorizing `identity:delete_project(project_id=ac770309087d4370844e15a4b62ae22b)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:14:42.032379 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3bad1e0-a9c4-4085-8986-24d17e7a5bab admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:14:42.103536 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-c3bad1e0-a9c4-4085-8986-24d17e7a5bab admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ac770309087d4370844e15a4b62ae22b', 'request_id': 'req-c3bad1e0-a9c4-4085-8986-24d17e7a5bab'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:14:42.111944 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-c3bad1e0-a9c4-4085-8986-24d17e7a5bab admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ac770309087d4370844e15a4b62ae22b', 'request_id': 'req-c3bad1e0-a9c4-4085-8986-24d17e7a5bab'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:14:42.119571 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-c3bad1e0-a9c4-4085-8986-24d17e7a5bab admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ac770309087d4370844e15a4b62ae22b', 'request_id': 'req-c3bad1e0-a9c4-4085-8986-24d17e7a5bab'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:14:42.136785 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-c3bad1e0-a9c4-4085-8986-24d17e7a5bab admin admin] The token cache is being invalidate because project ac770309087d4370844e15a4b62ae22b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80381) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 22:14:42.137150 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-c3bad1e0-a9c4-4085-8986-24d17e7a5bab admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:14:42.137716 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-c3bad1e0-a9c4-4085-8986-24d17e7a5bab admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:14:42.141043 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1089/2171] 10.222.0.52 () {58 vars in 1232 bytes} [Wed Jul 13 22:14:41 2022] DELETE /identity/v3/projects/ac770309087d4370844e15a4b62ae22b => generated 0 bytes in 214 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 22:14:42.147256 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad15790-2873-4f26-823b-853e7db36519 None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:14:42.147797 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad15790-2873-4f26-823b-853e7db36519 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:14:42.147797 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ad15790-2873-4f26-823b-853e7db36519 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:14:42.167561 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-5ad15790-2873-4f26-823b-853e7db36519 None None] MFA Rules not processed for user `cc99c93da5074cf5a48bc2aa9a92c5e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 22:14:42.170427 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-5ad15790-2873-4f26-823b-853e7db36519 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:14:42.202091 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:14:42.253751 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1083/2172] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jul 13 22:14:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 22:14:42.262763 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874d0a58-2d29-4245-9a23-4e727e3e175d None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:14:42.263050 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874d0a58-2d29-4245-9a23-4e727e3e175d None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:14:42.263375 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874d0a58-2d29-4245-9a23-4e727e3e175d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:14:42.287910 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-874d0a58-2d29-4245-9a23-4e727e3e175d None None] MFA Rules not processed for user `cc99c93da5074cf5a48bc2aa9a92c5e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 22:14:42.290464 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-874d0a58-2d29-4245-9a23-4e727e3e175d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:14:42.300386 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1090/2173] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jul 13 22:14:42 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 39 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 22:14:42.305832 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad15790-2873-4f26-823b-853e7db36519 None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:14:42.318869 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ad15790-2873-4f26-823b-853e7db36519 None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:14:42.324540 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41064ac-0c6f-409c-bb7b-29d01fc52404 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:14:42.325220 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41064ac-0c6f-409c-bb7b-29d01fc52404 admin admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:14:42.325493 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41064ac-0c6f-409c-bb7b-29d01fc52404 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:14:42.325768 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c41064ac-0c6f-409c-bb7b-29d01fc52404 admin admin] PATH_INFO: `/v3/domains` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:14:42.326536 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41064ac-0c6f-409c-bb7b-29d01fc52404 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 22:14:42.326864 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41064ac-0c6f-409c-bb7b-29d01fc52404 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:14:42.328513 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c41064ac-0c6f-409c-bb7b-29d01fc52404 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:14:42.341741 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1084/2174] 10.222.0.52 () {58 vars in 1152 bytes} [Wed Jul 13 22:14:42 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 22:14:42.349958 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874d0a58-2d29-4245-9a23-4e727e3e175d None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:14:42.362389 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874d0a58-2d29-4245-9a23-4e727e3e175d None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:14:42.369615 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7c56de-f755-481f-b21a-e2c576e09559 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:14:42.370833 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7c56de-f755-481f-b21a-e2c576e09559 admin admin] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:14:42.371217 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7c56de-f755-481f-b21a-e2c576e09559 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:14:42.371590 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e7c56de-f755-481f-b21a-e2c576e09559 admin admin] PATH_INFO: `/v3/projects` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:14:42.372633 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e7c56de-f755-481f-b21a-e2c576e09559 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:14:42.375008 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e7c56de-f755-481f-b21a-e2c576e09559 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:14:42.407141 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1091/2175] 10.222.0.52 () {60 vars in 1152 bytes} [Wed Jul 13 22:14:42 2022] POST /identity/v3/projects => generated 377 bytes in 58 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 22:14:42.411699 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41064ac-0c6f-409c-bb7b-29d01fc52404 admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:14:42.426258 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c41064ac-0c6f-409c-bb7b-29d01fc52404 admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:14:42.486316 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f1cfd8-47b8-45f4-bde8-ece8170b85e9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:14:42.487171 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f1cfd8-47b8-45f4-bde8-ece8170b85e9 admin admin] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:14:42.487448 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f1cfd8-47b8-45f4-bde8-ece8170b85e9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:14:42.487745 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4f1cfd8-47b8-45f4-bde8-ece8170b85e9 admin admin] PATH_INFO: `/v3/users` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:14:42.488698 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f1cfd8-47b8-45f4-bde8-ece8170b85e9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:14:42.490856 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4f1cfd8-47b8-45f4-bde8-ece8170b85e9 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:14:42.523354 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1085/2176] 10.222.0.52 () {60 vars in 1143 bytes} [Wed Jul 13 22:14:42 2022] POST /identity/v3/users => generated 289 bytes in 112 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 13 22:14:42.527746 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7c56de-f755-481f-b21a-e2c576e09559 admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:14:42.539482 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e7c56de-f755-481f-b21a-e2c576e09559 admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:14:42.544810 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdeb46c1-b31f-487d-8075-a761f46748d3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:14:42.545520 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdeb46c1-b31f-487d-8075-a761f46748d3 admin admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:14:42.545802 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdeb46c1-b31f-487d-8075-a761f46748d3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:14:42.546071 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdeb46c1-b31f-487d-8075-a761f46748d3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:14:42.546863 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdeb46c1-b31f-487d-8075-a761f46748d3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:14:42.549132 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-cdeb46c1-b31f-487d-8075-a761f46748d3 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:14:42.551725 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=83,keystone:DELETE=14,keystone:INSERT=3 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:14:42.562087 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1092/2177] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jul 13 22:14:42 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 22:14:42.566668 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f1cfd8-47b8-45f4-bde8-ece8170b85e9 admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:14:42.581899 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4f1cfd8-47b8-45f4-bde8-ece8170b85e9 admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:14:42.587488 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8315b845-6271-432a-b2c3-751f94a30836 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:14:42.588188 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8315b845-6271-432a-b2c3-751f94a30836 admin admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:14:42.588499 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8315b845-6271-432a-b2c3-751f94a30836 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:14:42.589327 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8315b845-6271-432a-b2c3-751f94a30836 admin admin] PATH_INFO: `/v3/roles` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:14:42.590711 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8315b845-6271-432a-b2c3-751f94a30836 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:14:42.593460 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8315b845-6271-432a-b2c3-751f94a30836 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:14:42.605084 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1086/2178] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jul 13 22:14:42 2022] GET /identity/v3/roles => generated 1420 bytes in 39 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 22:14:42.609882 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdeb46c1-b31f-487d-8075-a761f46748d3 admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:14:42.620767 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdeb46c1-b31f-487d-8075-a761f46748d3 admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:14:42.626052 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915ae18d-f922-4721-b58c-d9f2afab4b73 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:14:42.627041 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915ae18d-f922-4721-b58c-d9f2afab4b73 admin admin] REQUEST_METHOD: `PUT` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:14:42.627344 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915ae18d-f922-4721-b58c-d9f2afab4b73 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:14:42.627616 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-915ae18d-f922-4721-b58c-d9f2afab4b73 admin admin] PATH_INFO: `/v3/projects/6754f83bae8d4e1bb222156eeb9a1d4d/users/ba1ea8ec942e4b219a55f00315b510f5/roles/2a5e86976394498abb6310697cdb3f8b` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:14:42.643489 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915ae18d-f922-4721-b58c-d9f2afab4b73 admin admin] RBAC: Authorizing `identity:create_grant(project_id=6754f83bae8d4e1bb222156eeb9a1d4d, user_id=ba1ea8ec942e4b219a55f00315b510f5, role_id=2a5e86976394498abb6310697cdb3f8b)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:14:42.646684 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-915ae18d-f922-4721-b58c-d9f2afab4b73 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:14:42.661781 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1093/2179] 10.222.0.52 () {60 vars in 1482 bytes} [Wed Jul 13 22:14:42 2022] PUT /identity/v3/projects/6754f83bae8d4e1bb222156eeb9a1d4d/users/ba1ea8ec942e4b219a55f00315b510f5/roles/2a5e86976394498abb6310697cdb3f8b => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 22:14:42.673137 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98f508d-c472-4937-9b14-456540ba6de2 None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:14:42.673454 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98f508d-c472-4937-9b14-456540ba6de2 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:14:42.673733 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e98f508d-c472-4937-9b14-456540ba6de2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:14:42.702203 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-e98f508d-c472-4937-9b14-456540ba6de2 None None] MFA Rules not processed for user `ba1ea8ec942e4b219a55f00315b510f5`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 22:14:42.703633 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-e98f508d-c472-4937-9b14-456540ba6de2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:14:42.772910 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1087/2180] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jul 13 22:14:42 2022] POST /identity/v3/auth/tokens => generated 3244 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 13 22:14:42.777868 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915ae18d-f922-4721-b58c-d9f2afab4b73 admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:14:42.843897 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-915ae18d-f922-4721-b58c-d9f2afab4b73 admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:14:42.848723 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc4b58c-2452-4be2-a386-d28e04a1e5ef admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:14:42.849398 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc4b58c-2452-4be2-a386-d28e04a1e5ef admin admin] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:14:42.849679 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc4b58c-2452-4be2-a386-d28e04a1e5ef admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:14:42.849953 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6cc4b58c-2452-4be2-a386-d28e04a1e5ef admin admin] PATH_INFO: `/v3/projects` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:14:42.850720 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc4b58c-2452-4be2-a386-d28e04a1e5ef admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:14:42.852651 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-6cc4b58c-2452-4be2-a386-d28e04a1e5ef admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:14:42.879924 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1094/2181] 10.222.0.52 () {60 vars in 1152 bytes} [Wed Jul 13 22:14:42 2022] POST /identity/v3/projects => generated 379 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 22:14:42.883883 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98f508d-c472-4937-9b14-456540ba6de2 None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:14:42.894931 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e98f508d-c472-4937-9b14-456540ba6de2 None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:14:42.919685 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:14:42.957761 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c29386-c119-4a43-ad08-d15af90ebfa4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:14:42.958453 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c29386-c119-4a43-ad08-d15af90ebfa4 admin admin] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:14:42.958769 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c29386-c119-4a43-ad08-d15af90ebfa4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:14:42.959052 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2c29386-c119-4a43-ad08-d15af90ebfa4 admin admin] PATH_INFO: `/v3/users` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:14:42.959772 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c29386-c119-4a43-ad08-d15af90ebfa4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:14:42.961707 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c2c29386-c119-4a43-ad08-d15af90ebfa4 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:14:42.982522 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1088/2182] 10.222.0.52 () {60 vars in 1143 bytes} [Wed Jul 13 22:14:42 2022] POST /identity/v3/users => generated 290 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 22:14:42.986953 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc4b58c-2452-4be2-a386-d28e04a1e5ef admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:14:42.998391 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6cc4b58c-2452-4be2-a386-d28e04a1e5ef admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:14:43.003834 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0cc7a0e-7bc1-4ed8-abe3-21c46fc0f56b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:14:43.004506 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cc7a0e-7bc1-4ed8-abe3-21c46fc0f56b admin admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:14:43.004785 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cc7a0e-7bc1-4ed8-abe3-21c46fc0f56b admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:14:43.005173 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b0cc7a0e-7bc1-4ed8-abe3-21c46fc0f56b admin admin] PATH_INFO: `/v3/roles` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:14:43.005935 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0cc7a0e-7bc1-4ed8-abe3-21c46fc0f56b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:14:43.007585 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b0cc7a0e-7bc1-4ed8-abe3-21c46fc0f56b admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:14:43.019033 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1095/2183] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jul 13 22:14:42 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 22:14:43.023785 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c29386-c119-4a43-ad08-d15af90ebfa4 admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:14:43.034171 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2c29386-c119-4a43-ad08-d15af90ebfa4 admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:14:43.039163 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a84c3df7-be5d-49c0-9107-43eb1760f101 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:14:43.039828 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84c3df7-be5d-49c0-9107-43eb1760f101 admin admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:14:43.040108 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84c3df7-be5d-49c0-9107-43eb1760f101 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:14:43.040383 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a84c3df7-be5d-49c0-9107-43eb1760f101 admin admin] PATH_INFO: `/v3/roles` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:14:43.041121 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84c3df7-be5d-49c0-9107-43eb1760f101 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:14:43.042772 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a84c3df7-be5d-49c0-9107-43eb1760f101 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:14:43.053020 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1089/2184] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jul 13 22:14:43 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 13 22:14:43.057869 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0cc7a0e-7bc1-4ed8-abe3-21c46fc0f56b admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:14:43.071192 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b0cc7a0e-7bc1-4ed8-abe3-21c46fc0f56b admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:14:43.077572 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ef778b-19af-4f26-ba73-c0197770fe6d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:14:43.078469 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ef778b-19af-4f26-ba73-c0197770fe6d admin admin] REQUEST_METHOD: `PUT` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:14:43.078819 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ef778b-19af-4f26-ba73-c0197770fe6d admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:14:43.079050 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-63ef778b-19af-4f26-ba73-c0197770fe6d admin admin] PATH_INFO: `/v3/projects/8f809c086a8f4fcd8acaf91593a208e8/users/66972ae114a7493abb9b87c9121b1608/roles/2a5e86976394498abb6310697cdb3f8b` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:14:43.094794 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ef778b-19af-4f26-ba73-c0197770fe6d admin admin] RBAC: Authorizing `identity:create_grant(project_id=8f809c086a8f4fcd8acaf91593a208e8, user_id=66972ae114a7493abb9b87c9121b1608, role_id=2a5e86976394498abb6310697cdb3f8b)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:14:43.097512 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-63ef778b-19af-4f26-ba73-c0197770fe6d admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:14:43.112119 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1096/2185] 10.222.0.52 () {60 vars in 1482 bytes} [Wed Jul 13 22:14:43 2022] PUT /identity/v3/projects/8f809c086a8f4fcd8acaf91593a208e8/users/66972ae114a7493abb9b87c9121b1608/roles/2a5e86976394498abb6310697cdb3f8b => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 13 22:14:43.122754 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2926abde-ae89-4999-8440-12a5750f8b06 None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:14:43.123107 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2926abde-ae89-4999-8440-12a5750f8b06 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:14:43.123471 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2926abde-ae89-4999-8440-12a5750f8b06 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:14:43.150666 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-2926abde-ae89-4999-8440-12a5750f8b06 None None] MFA Rules not processed for user `66972ae114a7493abb9b87c9121b1608`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 22:14:43.151861 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-2926abde-ae89-4999-8440-12a5750f8b06 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:14:43.217838 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1090/2186] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jul 13 22:14:43 2022] POST /identity/v3/auth/tokens => generated 3246 bytes in 96 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 13 22:14:43.222393 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ef778b-19af-4f26-ba73-c0197770fe6d admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:14:43.287847 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-63ef778b-19af-4f26-ba73-c0197770fe6d admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:14:43.293057 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d82f94a-04b3-469f-bfb2-67258071b3c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:14:43.293707 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d82f94a-04b3-469f-bfb2-67258071b3c1 admin admin] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:14:43.293989 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d82f94a-04b3-469f-bfb2-67258071b3c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:14:43.294257 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3d82f94a-04b3-469f-bfb2-67258071b3c1 admin admin] PATH_INFO: `/v3/projects` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:14:43.295006 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d82f94a-04b3-469f-bfb2-67258071b3c1 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:14:43.296920 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3d82f94a-04b3-469f-bfb2-67258071b3c1 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:14:43.325402 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1097/2187] 10.222.0.52 () {60 vars in 1152 bytes} [Wed Jul 13 22:14:43 2022] POST /identity/v3/projects => generated 379 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 22:14:43.329681 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2926abde-ae89-4999-8440-12a5750f8b06 None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:14:43.341455 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2926abde-ae89-4999-8440-12a5750f8b06 None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:14:43.400725 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa999776-9c49-4f07-83f8-f22b245f87a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:14:43.401523 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa999776-9c49-4f07-83f8-f22b245f87a3 admin admin] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:14:43.401875 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa999776-9c49-4f07-83f8-f22b245f87a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:14:43.402159 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa999776-9c49-4f07-83f8-f22b245f87a3 admin admin] PATH_INFO: `/v3/users` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:14:43.402929 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa999776-9c49-4f07-83f8-f22b245f87a3 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:14:43.404872 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa999776-9c49-4f07-83f8-f22b245f87a3 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:14:43.427186 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1091/2188] 10.222.0.52 () {60 vars in 1143 bytes} [Wed Jul 13 22:14:43 2022] POST /identity/v3/users => generated 296 bytes in 98 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 22:14:43.431344 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d82f94a-04b3-469f-bfb2-67258071b3c1 admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:14:43.441694 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3d82f94a-04b3-469f-bfb2-67258071b3c1 admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:14:43.446291 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47af6b6f-a286-4c94-add0-145d29fb0377 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:14:43.447322 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47af6b6f-a286-4c94-add0-145d29fb0377 admin admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:14:43.447603 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47af6b6f-a286-4c94-add0-145d29fb0377 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:14:43.447876 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47af6b6f-a286-4c94-add0-145d29fb0377 admin admin] PATH_INFO: `/v3/roles` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:14:43.448638 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47af6b6f-a286-4c94-add0-145d29fb0377 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:14:43.450238 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47af6b6f-a286-4c94-add0-145d29fb0377 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:14:43.452264 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=96,keystone:INSERT=4 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:14:43.462682 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1098/2189] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jul 13 22:14:43 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 22:14:43.466512 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa999776-9c49-4f07-83f8-f22b245f87a3 admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:14:43.477535 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa999776-9c49-4f07-83f8-f22b245f87a3 admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:14:43.482464 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6765b1-9bf1-4250-9909-9c5b5c96cdab admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:14:43.483392 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6765b1-9bf1-4250-9909-9c5b5c96cdab admin admin] REQUEST_METHOD: `PUT` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:14:43.483663 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6765b1-9bf1-4250-9909-9c5b5c96cdab admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:14:43.483930 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e6765b1-9bf1-4250-9909-9c5b5c96cdab admin admin] PATH_INFO: `/v3/projects/8fd5f6d3686d438198c8b16940586e9b/users/05e35dca6eaa457fbb2a527d9d70817b/roles/44efba2579f342df8a0789c59d163ec3` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:14:43.500326 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e6765b1-9bf1-4250-9909-9c5b5c96cdab admin admin] RBAC: Authorizing `identity:create_grant(project_id=8fd5f6d3686d438198c8b16940586e9b, user_id=05e35dca6eaa457fbb2a527d9d70817b, role_id=44efba2579f342df8a0789c59d163ec3)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:14:43.503256 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e6765b1-9bf1-4250-9909-9c5b5c96cdab admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:14:43.513000 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=93,keystone:INSERT=7 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:14:43.518294 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1092/2190] 10.222.0.52 () {60 vars in 1482 bytes} [Wed Jul 13 22:14:43 2022] PUT /identity/v3/projects/8fd5f6d3686d438198c8b16940586e9b/users/05e35dca6eaa457fbb2a527d9d70817b/roles/44efba2579f342df8a0789c59d163ec3 => generated 0 bytes in 52 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 22:14:43.522948 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47af6b6f-a286-4c94-add0-145d29fb0377 admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:14:43.533347 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47af6b6f-a286-4c94-add0-145d29fb0377 admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:14:43.599697 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ec01d61-a441-4f3d-86c3-531500eedf50 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:14:43.600376 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec01d61-a441-4f3d-86c3-531500eedf50 admin admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:14:43.600700 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec01d61-a441-4f3d-86c3-531500eedf50 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:14:43.600970 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ec01d61-a441-4f3d-86c3-531500eedf50 admin admin] PATH_INFO: `/v3/roles` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:14:43.601707 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec01d61-a441-4f3d-86c3-531500eedf50 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:14:43.603374 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ec01d61-a441-4f3d-86c3-531500eedf50 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:14:43.613817 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1099/2191] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jul 13 22:14:43 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 22:14:43.618255 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6765b1-9bf1-4250-9909-9c5b5c96cdab admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:14:43.632615 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e6765b1-9bf1-4250-9909-9c5b5c96cdab admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:14:43.638754 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ab8ff4-2ea4-4e1e-bf0f-70da3466c80c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:14:43.639650 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ab8ff4-2ea4-4e1e-bf0f-70da3466c80c admin admin] REQUEST_METHOD: `PUT` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:14:43.639929 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ab8ff4-2ea4-4e1e-bf0f-70da3466c80c admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:14:43.640198 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c9ab8ff4-2ea4-4e1e-bf0f-70da3466c80c admin admin] PATH_INFO: `/v3/projects/8fd5f6d3686d438198c8b16940586e9b/users/05e35dca6eaa457fbb2a527d9d70817b/roles/44efba2579f342df8a0789c59d163ec3` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:14:43.642771 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ab8ff4-2ea4-4e1e-bf0f-70da3466c80c admin admin] RBAC: Authorizing `identity:create_grant(project_id=8fd5f6d3686d438198c8b16940586e9b, user_id=05e35dca6eaa457fbb2a527d9d70817b, role_id=44efba2579f342df8a0789c59d163ec3)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:14:43.645505 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c9ab8ff4-2ea4-4e1e-bf0f-70da3466c80c admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:14:43.656674 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1093/2192] 10.222.0.52 () {60 vars in 1482 bytes} [Wed Jul 13 22:14:43 2022] PUT /identity/v3/projects/8fd5f6d3686d438198c8b16940586e9b/users/05e35dca6eaa457fbb2a527d9d70817b/roles/44efba2579f342df8a0789c59d163ec3 => generated 0 bytes in 39 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 22:14:43.668376 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe15a64-5f1f-45da-b725-acdad310aebf None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:14:43.668664 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe15a64-5f1f-45da-b725-acdad310aebf None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:14:43.668938 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ffe15a64-5f1f-45da-b725-acdad310aebf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:14:43.696592 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-ffe15a64-5f1f-45da-b725-acdad310aebf None None] MFA Rules not processed for user `05e35dca6eaa457fbb2a527d9d70817b`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 22:14:43.698172 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-ffe15a64-5f1f-45da-b725-acdad310aebf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:14:43.780610 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1100/2193] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jul 13 22:14:43 2022] POST /identity/v3/auth/tokens => generated 3313 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 22:14:43.792342 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ab8ff4-2ea4-4e1e-bf0f-70da3466c80c admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:14:43.794202 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-c9ab8ff4-2ea4-4e1e-bf0f-70da3466c80c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:14:43.861558 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c9ab8ff4-2ea4-4e1e-bf0f-70da3466c80c admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:14:43.866292 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5a8982-01dd-4bfd-97c5-0050efa64879 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc77e1ea1ada4700b809beb1c980c5e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417e3e2f1333481795a089bef2e24bea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:14:43.866988 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5a8982-01dd-4bfd-97c5-0050efa64879 service cinder] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:14:43.867277 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5a8982-01dd-4bfd-97c5-0050efa64879 service cinder] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:14:43.867540 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fb5a8982-01dd-4bfd-97c5-0050efa64879 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:14:43.878125 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb5a8982-01dd-4bfd-97c5-0050efa64879 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:14:43.879637 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fb5a8982-01dd-4bfd-97c5-0050efa64879 service cinder] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:14:43.889930 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1094/2194] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jul 13 22:14:43 2022] GET /identity/v3/auth/tokens => generated 3313 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 22:14:43.932380 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe15a64-5f1f-45da-b725-acdad310aebf None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:14:43.934975 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-ffe15a64-5f1f-45da-b725-acdad310aebf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:14:43.949769 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ffe15a64-5f1f-45da-b725-acdad310aebf None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:14:43.956200 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2dcc4c94-9518-4af6-95d7-f78db8807b9d service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc77e1ea1ada4700b809beb1c980c5e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417e3e2f1333481795a089bef2e24bea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:14:43.956868 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcc4c94-9518-4af6-95d7-f78db8807b9d service cinder] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:14:43.957144 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcc4c94-9518-4af6-95d7-f78db8807b9d service cinder] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:14:43.957410 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2dcc4c94-9518-4af6-95d7-f78db8807b9d service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:14:44.018100 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dcc4c94-9518-4af6-95d7-f78db8807b9d service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:14:44.019569 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2dcc4c94-9518-4af6-95d7-f78db8807b9d service cinder] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:14:44.028849 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1101/2195] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jul 13 22:14:43 2022] GET /identity/v3/auth/tokens => generated 3244 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 22:14:48.010677 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5a8982-01dd-4bfd-97c5-0050efa64879 service cinder] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:14:48.079249 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fb5a8982-01dd-4bfd-97c5-0050efa64879 service cinder] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:14:48.084221 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931fa021-9e5e-459c-91f6-841640e75954 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:14:48.085108 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931fa021-9e5e-459c-91f6-841640e75954 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:14:48.085388 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931fa021-9e5e-459c-91f6-841640e75954 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:14:48.085662 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-931fa021-9e5e-459c-91f6-841640e75954 admin admin] PATH_INFO: `/v3/users/ba1ea8ec942e4b219a55f00315b510f5` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:14:48.086991 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-931fa021-9e5e-459c-91f6-841640e75954 admin admin] RBAC: Authorizing `identity:delete_user(user_id=ba1ea8ec942e4b219a55f00315b510f5)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:14:48.088990 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-931fa021-9e5e-459c-91f6-841640e75954 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:14:48.141294 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-931fa021-9e5e-459c-91f6-841640e75954 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'ba1ea8ec942e4b219a55f00315b510f5'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:14:48.149252 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-931fa021-9e5e-459c-91f6-841640e75954 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'ba1ea8ec942e4b219a55f00315b510f5'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:14:48.162060 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-931fa021-9e5e-459c-91f6-841640e75954 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'ba1ea8ec942e4b219a55f00315b510f5'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:14:48.162667 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-931fa021-9e5e-459c-91f6-841640e75954 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'ba1ea8ec942e4b219a55f00315b510f5'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:14:48.163289 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-931fa021-9e5e-459c-91f6-841640e75954 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'ba1ea8ec942e4b219a55f00315b510f5'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:14:48.176412 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1095/2196] 10.222.0.52 () {58 vars in 1223 bytes} [Wed Jul 13 22:14:48 2022] DELETE /identity/v3/users/ba1ea8ec942e4b219a55f00315b510f5 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 22:14:48.182146 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681efefc-87c2-468e-a4aa-22e1f40ab1ff None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:14:48.182666 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681efefc-87c2-468e-a4aa-22e1f40ab1ff None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:14:48.182759 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-681efefc-87c2-468e-a4aa-22e1f40ab1ff None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:14:48.201555 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-681efefc-87c2-468e-a4aa-22e1f40ab1ff None None] MFA Rules not processed for user `cc99c93da5074cf5a48bc2aa9a92c5e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 22:14:48.202787 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-681efefc-87c2-468e-a4aa-22e1f40ab1ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:14:48.215725 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:14:48.265357 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1102/2197] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jul 13 22:14:48 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 84 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 22:14:48.275826 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931fa021-9e5e-459c-91f6-841640e75954 admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:14:48.277446 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-931fa021-9e5e-459c-91f6-841640e75954 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:14:48.289606 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=88,keystone:INSERT=2,keystone:DELETE=10 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:14:48.342499 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-931fa021-9e5e-459c-91f6-841640e75954 admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:14:48.346756 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6381c2-d9f6-4a98-8571-b8096e0a9cb7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '847ec9a1bd4d4dec804318a5c845895a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417e3e2f1333481795a089bef2e24bea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:14:48.347413 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6381c2-d9f6-4a98-8571-b8096e0a9cb7 service neutron] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:14:48.347687 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6381c2-d9f6-4a98-8571-b8096e0a9cb7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:14:48.347955 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc6381c2-d9f6-4a98-8571-b8096e0a9cb7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:14:48.359024 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6381c2-d9f6-4a98-8571-b8096e0a9cb7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:14:48.360519 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bc6381c2-d9f6-4a98-8571-b8096e0a9cb7 service neutron] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:14:48.370523 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1096/2198] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jul 13 22:14:48 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 22:14:48.653766 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681efefc-87c2-468e-a4aa-22e1f40ab1ff None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:14:48.655329 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-681efefc-87c2-468e-a4aa-22e1f40ab1ff None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:14:48.669312 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-681efefc-87c2-468e-a4aa-22e1f40ab1ff None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:14:48.675113 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4e9d18-9b04-41f9-b721-8a4009a34d26 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:14:48.676128 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4e9d18-9b04-41f9-b721-8a4009a34d26 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:14:48.676403 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4e9d18-9b04-41f9-b721-8a4009a34d26 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:14:48.676743 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ca4e9d18-9b04-41f9-b721-8a4009a34d26 admin admin] PATH_INFO: `/v3/projects/6754f83bae8d4e1bb222156eeb9a1d4d` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:14:48.678664 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4e9d18-9b04-41f9-b721-8a4009a34d26 admin admin] RBAC: Authorizing `identity:delete_project(project_id=6754f83bae8d4e1bb222156eeb9a1d4d)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:14:48.681099 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ca4e9d18-9b04-41f9-b721-8a4009a34d26 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:14:48.733576 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-ca4e9d18-9b04-41f9-b721-8a4009a34d26 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6754f83bae8d4e1bb222156eeb9a1d4d', 'request_id': 'req-ca4e9d18-9b04-41f9-b721-8a4009a34d26'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:14:48.740330 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-ca4e9d18-9b04-41f9-b721-8a4009a34d26 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6754f83bae8d4e1bb222156eeb9a1d4d', 'request_id': 'req-ca4e9d18-9b04-41f9-b721-8a4009a34d26'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:14:48.746331 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-ca4e9d18-9b04-41f9-b721-8a4009a34d26 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6754f83bae8d4e1bb222156eeb9a1d4d', 'request_id': 'req-ca4e9d18-9b04-41f9-b721-8a4009a34d26'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:14:48.762852 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-ca4e9d18-9b04-41f9-b721-8a4009a34d26 admin admin] The token cache is being invalidate because project 6754f83bae8d4e1bb222156eeb9a1d4d was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80381) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 22:14:48.763132 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-ca4e9d18-9b04-41f9-b721-8a4009a34d26 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:14:48.763756 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-ca4e9d18-9b04-41f9-b721-8a4009a34d26 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:14:48.765359 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1103/2199] 10.222.0.52 () {58 vars in 1232 bytes} [Wed Jul 13 22:14:48 2022] DELETE /identity/v3/projects/6754f83bae8d4e1bb222156eeb9a1d4d => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 22:14:48.769538 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6381c2-d9f6-4a98-8571-b8096e0a9cb7 service neutron] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:14:48.779534 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc6381c2-d9f6-4a98-8571-b8096e0a9cb7 service neutron] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:14:48.843069 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b61c92-378e-4561-a2c5-958a80b7da04 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:14:48.843982 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b61c92-378e-4561-a2c5-958a80b7da04 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:14:48.844383 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b61c92-378e-4561-a2c5-958a80b7da04 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:14:48.844656 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f6b61c92-378e-4561-a2c5-958a80b7da04 admin admin] PATH_INFO: `/v3/users/66972ae114a7493abb9b87c9121b1608` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:14:48.845878 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b61c92-378e-4561-a2c5-958a80b7da04 admin admin] RBAC: Authorizing `identity:delete_user(user_id=66972ae114a7493abb9b87c9121b1608)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:14:48.848013 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f6b61c92-378e-4561-a2c5-958a80b7da04 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:14:48.899064 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-f6b61c92-378e-4561-a2c5-958a80b7da04 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '66972ae114a7493abb9b87c9121b1608'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:14:48.908166 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-f6b61c92-378e-4561-a2c5-958a80b7da04 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '66972ae114a7493abb9b87c9121b1608'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:14:48.921756 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-f6b61c92-378e-4561-a2c5-958a80b7da04 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '66972ae114a7493abb9b87c9121b1608'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:14:48.922575 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-f6b61c92-378e-4561-a2c5-958a80b7da04 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '66972ae114a7493abb9b87c9121b1608'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:14:48.923263 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-f6b61c92-378e-4561-a2c5-958a80b7da04 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '66972ae114a7493abb9b87c9121b1608'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:14:48.935460 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1097/2200] 10.222.0.52 () {58 vars in 1223 bytes} [Wed Jul 13 22:14:48 2022] DELETE /identity/v3/users/66972ae114a7493abb9b87c9121b1608 => generated 0 bytes in 167 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 13 22:14:49.298406 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4e9d18-9b04-41f9-b721-8a4009a34d26 admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:14:49.300137 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-ca4e9d18-9b04-41f9-b721-8a4009a34d26 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:14:49.315761 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ca4e9d18-9b04-41f9-b721-8a4009a34d26 admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:14:49.375017 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed41bbf1-4d03-4653-a6a3-34a8666ba7a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:14:49.375891 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed41bbf1-4d03-4653-a6a3-34a8666ba7a3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:14:49.376167 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed41bbf1-4d03-4653-a6a3-34a8666ba7a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:14:49.376439 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ed41bbf1-4d03-4653-a6a3-34a8666ba7a3 admin admin] PATH_INFO: `/v3/projects/8f809c086a8f4fcd8acaf91593a208e8` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:14:49.377639 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed41bbf1-4d03-4653-a6a3-34a8666ba7a3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=8f809c086a8f4fcd8acaf91593a208e8)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:14:49.379592 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ed41bbf1-4d03-4653-a6a3-34a8666ba7a3 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:14:49.429928 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:14:49.434273 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-ed41bbf1-4d03-4653-a6a3-34a8666ba7a3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8f809c086a8f4fcd8acaf91593a208e8', 'request_id': 'req-ed41bbf1-4d03-4653-a6a3-34a8666ba7a3'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:14:49.441569 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-ed41bbf1-4d03-4653-a6a3-34a8666ba7a3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8f809c086a8f4fcd8acaf91593a208e8', 'request_id': 'req-ed41bbf1-4d03-4653-a6a3-34a8666ba7a3'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:14:49.448863 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-ed41bbf1-4d03-4653-a6a3-34a8666ba7a3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8f809c086a8f4fcd8acaf91593a208e8', 'request_id': 'req-ed41bbf1-4d03-4653-a6a3-34a8666ba7a3'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:14:49.463328 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-ed41bbf1-4d03-4653-a6a3-34a8666ba7a3 admin admin] The token cache is being invalidate because project 8f809c086a8f4fcd8acaf91593a208e8 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80381) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 22:14:49.463611 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-ed41bbf1-4d03-4653-a6a3-34a8666ba7a3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:14:49.464234 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-ed41bbf1-4d03-4653-a6a3-34a8666ba7a3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:14:49.465788 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1104/2201] 10.222.0.52 () {58 vars in 1232 bytes} [Wed Jul 13 22:14:49 2022] DELETE /identity/v3/projects/8f809c086a8f4fcd8acaf91593a208e8 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 22:14:49.470008 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b61c92-378e-4561-a2c5-958a80b7da04 admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:14:49.471479 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-f6b61c92-378e-4561-a2c5-958a80b7da04 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:14:49.484208 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f6b61c92-378e-4561-a2c5-958a80b7da04 admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:14:49.512674 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:14:49.542910 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4a16596-2dfa-4eea-8be6-5bbde35168ff admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:14:49.543722 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a16596-2dfa-4eea-8be6-5bbde35168ff admin admin] REQUEST_METHOD: `DELETE` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:14:49.543995 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a16596-2dfa-4eea-8be6-5bbde35168ff admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:14:49.544266 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4a16596-2dfa-4eea-8be6-5bbde35168ff admin admin] PATH_INFO: `/v3/users/05e35dca6eaa457fbb2a527d9d70817b` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:14:49.545449 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a16596-2dfa-4eea-8be6-5bbde35168ff admin admin] RBAC: Authorizing `identity:delete_user(user_id=05e35dca6eaa457fbb2a527d9d70817b)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:14:49.547412 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4a16596-2dfa-4eea-8be6-5bbde35168ff admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:14:49.604378 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-f4a16596-2dfa-4eea-8be6-5bbde35168ff admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '05e35dca6eaa457fbb2a527d9d70817b'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:14:49.615186 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-f4a16596-2dfa-4eea-8be6-5bbde35168ff admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '05e35dca6eaa457fbb2a527d9d70817b'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:14:49.627499 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-f4a16596-2dfa-4eea-8be6-5bbde35168ff admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '05e35dca6eaa457fbb2a527d9d70817b'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:14:49.628137 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-f4a16596-2dfa-4eea-8be6-5bbde35168ff admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '05e35dca6eaa457fbb2a527d9d70817b'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:14:49.628745 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-f4a16596-2dfa-4eea-8be6-5bbde35168ff admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '05e35dca6eaa457fbb2a527d9d70817b'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:14:49.641472 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1098/2202] 10.222.0.52 () {58 vars in 1223 bytes} [Wed Jul 13 22:14:49 2022] DELETE /identity/v3/users/05e35dca6eaa457fbb2a527d9d70817b => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 22:14:49.914390 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed41bbf1-4d03-4653-a6a3-34a8666ba7a3 admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:14:49.916056 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-ed41bbf1-4d03-4653-a6a3-34a8666ba7a3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:14:49.930794 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ed41bbf1-4d03-4653-a6a3-34a8666ba7a3 admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:14:49.989677 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e889404e-9864-4136-a507-90e32f15f1ed admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:14:49.990670 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e889404e-9864-4136-a507-90e32f15f1ed admin admin] REQUEST_METHOD: `DELETE` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:14:49.990949 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e889404e-9864-4136-a507-90e32f15f1ed admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:14:49.991224 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e889404e-9864-4136-a507-90e32f15f1ed admin admin] PATH_INFO: `/v3/projects/8fd5f6d3686d438198c8b16940586e9b` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:14:49.992393 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e889404e-9864-4136-a507-90e32f15f1ed admin admin] RBAC: Authorizing `identity:delete_project(project_id=8fd5f6d3686d438198c8b16940586e9b)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:14:49.994306 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e889404e-9864-4136-a507-90e32f15f1ed admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:14:50.044941 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-e889404e-9864-4136-a507-90e32f15f1ed admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '8fd5f6d3686d438198c8b16940586e9b', 'request_id': 'req-e889404e-9864-4136-a507-90e32f15f1ed'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:14:50.051896 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-e889404e-9864-4136-a507-90e32f15f1ed admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '8fd5f6d3686d438198c8b16940586e9b', 'request_id': 'req-e889404e-9864-4136-a507-90e32f15f1ed'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:14:50.062585 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-e889404e-9864-4136-a507-90e32f15f1ed admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '8fd5f6d3686d438198c8b16940586e9b', 'request_id': 'req-e889404e-9864-4136-a507-90e32f15f1ed'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:14:50.077328 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-e889404e-9864-4136-a507-90e32f15f1ed admin admin] The token cache is being invalidate because project 8fd5f6d3686d438198c8b16940586e9b was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80381) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 22:14:50.077605 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-e889404e-9864-4136-a507-90e32f15f1ed admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:14:50.078237 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-e889404e-9864-4136-a507-90e32f15f1ed admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:14:50.079985 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1105/2203] 10.222.0.52 () {58 vars in 1232 bytes} [Wed Jul 13 22:14:49 2022] DELETE /identity/v3/projects/8fd5f6d3686d438198c8b16940586e9b => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 22:14:50.086155 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf4acc5-a665-4c66-98ac-ee3d9b7c25e4 None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:14:50.086370 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf4acc5-a665-4c66-98ac-ee3d9b7c25e4 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:14:50.086675 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-cdf4acc5-a665-4c66-98ac-ee3d9b7c25e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:14:50.104875 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-cdf4acc5-a665-4c66-98ac-ee3d9b7c25e4 None None] MFA Rules not processed for user `cc99c93da5074cf5a48bc2aa9a92c5e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 22:14:50.106108 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-cdf4acc5-a665-4c66-98ac-ee3d9b7c25e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:14:50.183120 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1099/2204] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jul 13 22:14:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 22:14:50.191604 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3d660f-b890-446b-af5d-31145bd12d56 None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:14:50.191890 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3d660f-b890-446b-af5d-31145bd12d56 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:14:50.192151 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2f3d660f-b890-446b-af5d-31145bd12d56 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:14:50.212456 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-2f3d660f-b890-446b-af5d-31145bd12d56 None None] MFA Rules not processed for user `cc99c93da5074cf5a48bc2aa9a92c5e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 22:14:50.213670 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-2f3d660f-b890-446b-af5d-31145bd12d56 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:14:50.222830 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1106/2205] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jul 13 22:14:50 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 22:14:50.228014 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf4acc5-a665-4c66-98ac-ee3d9b7c25e4 None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:14:50.238834 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-cdf4acc5-a665-4c66-98ac-ee3d9b7c25e4 None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:14:50.244193 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2757bfa-215b-422d-a5f4-b06c9cd95657 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:14:50.244850 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2757bfa-215b-422d-a5f4-b06c9cd95657 admin admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:14:50.245150 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2757bfa-215b-422d-a5f4-b06c9cd95657 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:14:50.245418 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d2757bfa-215b-422d-a5f4-b06c9cd95657 admin admin] PATH_INFO: `/v3/domains` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:14:50.246156 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2757bfa-215b-422d-a5f4-b06c9cd95657 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 22:14:50.246462 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2757bfa-215b-422d-a5f4-b06c9cd95657 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:14:50.248100 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d2757bfa-215b-422d-a5f4-b06c9cd95657 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:14:50.260448 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1100/2206] 10.222.0.52 () {58 vars in 1152 bytes} [Wed Jul 13 22:14:50 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 22:14:50.264856 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3d660f-b890-446b-af5d-31145bd12d56 None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:14:50.275117 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2f3d660f-b890-446b-af5d-31145bd12d56 None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:14:50.282644 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a700e44-66a9-4ef3-809a-4a235400a938 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:14:50.283696 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a700e44-66a9-4ef3-809a-4a235400a938 admin admin] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:14:50.284185 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a700e44-66a9-4ef3-809a-4a235400a938 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:14:50.284671 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0a700e44-66a9-4ef3-809a-4a235400a938 admin admin] PATH_INFO: `/v3/projects` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:14:50.285882 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a700e44-66a9-4ef3-809a-4a235400a938 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:14:50.288894 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0a700e44-66a9-4ef3-809a-4a235400a938 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:14:50.321227 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1107/2207] 10.222.0.52 () {60 vars in 1152 bytes} [Wed Jul 13 22:14:50 2022] POST /identity/v3/projects => generated 365 bytes in 57 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 22:14:50.326094 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2757bfa-215b-422d-a5f4-b06c9cd95657 admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:14:50.336662 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d2757bfa-215b-422d-a5f4-b06c9cd95657 admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:14:50.371450 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:14:50.409446 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd08ce6b-a5ae-4fb1-9ce5-5fa8196f053e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:14:50.410148 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd08ce6b-a5ae-4fb1-9ce5-5fa8196f053e admin admin] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:14:50.410432 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd08ce6b-a5ae-4fb1-9ce5-5fa8196f053e admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:14:50.410740 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bd08ce6b-a5ae-4fb1-9ce5-5fa8196f053e admin admin] PATH_INFO: `/v3/users` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:14:50.411465 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd08ce6b-a5ae-4fb1-9ce5-5fa8196f053e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:14:50.413474 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bd08ce6b-a5ae-4fb1-9ce5-5fa8196f053e admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:14:50.435290 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1101/2208] 10.222.0.52 () {60 vars in 1143 bytes} [Wed Jul 13 22:14:50 2022] POST /identity/v3/users => generated 283 bytes in 110 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 22:14:50.439277 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a700e44-66a9-4ef3-809a-4a235400a938 admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:14:50.450614 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0a700e44-66a9-4ef3-809a-4a235400a938 admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:14:50.455419 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d41f5c0-a87a-47fd-9ac6-f30bbc1ce68d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:14:50.456032 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d41f5c0-a87a-47fd-9ac6-f30bbc1ce68d admin admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:14:50.456311 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d41f5c0-a87a-47fd-9ac6-f30bbc1ce68d admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:14:50.456605 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d41f5c0-a87a-47fd-9ac6-f30bbc1ce68d admin admin] PATH_INFO: `/v3/roles` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:14:50.457335 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d41f5c0-a87a-47fd-9ac6-f30bbc1ce68d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:14:50.459055 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d41f5c0-a87a-47fd-9ac6-f30bbc1ce68d admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:14:50.470522 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1108/2209] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jul 13 22:14:50 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 22:14:50.473758 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd08ce6b-a5ae-4fb1-9ce5-5fa8196f053e admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:14:50.484224 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bd08ce6b-a5ae-4fb1-9ce5-5fa8196f053e admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:14:50.488841 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0e87a432-f5a2-4ec0-ae3c-45341c3bf34b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:14:50.489526 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e87a432-f5a2-4ec0-ae3c-45341c3bf34b admin admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:14:50.489803 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e87a432-f5a2-4ec0-ae3c-45341c3bf34b admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:14:50.490076 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0e87a432-f5a2-4ec0-ae3c-45341c3bf34b admin admin] PATH_INFO: `/v3/roles` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:14:50.490953 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e87a432-f5a2-4ec0-ae3c-45341c3bf34b admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:14:50.492539 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0e87a432-f5a2-4ec0-ae3c-45341c3bf34b admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:14:50.502755 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1102/2210] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jul 13 22:14:50 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 22:14:50.507171 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d41f5c0-a87a-47fd-9ac6-f30bbc1ce68d admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:14:50.517859 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d41f5c0-a87a-47fd-9ac6-f30bbc1ce68d admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:14:50.523041 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7b0046-f7b8-42d3-a3b6-4e35410aff6b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:14:50.523963 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7b0046-f7b8-42d3-a3b6-4e35410aff6b admin admin] REQUEST_METHOD: `PUT` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:14:50.524311 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7b0046-f7b8-42d3-a3b6-4e35410aff6b admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:14:50.524669 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc7b0046-f7b8-42d3-a3b6-4e35410aff6b admin admin] PATH_INFO: `/v3/projects/ec4d654647ad4d35ba6475136226be74/users/97f3be025adf4ffeacdfcb212e72e168/roles/2a5e86976394498abb6310697cdb3f8b` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:14:50.541188 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7b0046-f7b8-42d3-a3b6-4e35410aff6b admin admin] RBAC: Authorizing `identity:create_grant(project_id=ec4d654647ad4d35ba6475136226be74, user_id=97f3be025adf4ffeacdfcb212e72e168, role_id=2a5e86976394498abb6310697cdb3f8b)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:14:50.544139 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc7b0046-f7b8-42d3-a3b6-4e35410aff6b admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:14:50.553536 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=85,keystone:DELETE=11,keystone:INSERT=4 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:14:50.560503 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1109/2211] 10.222.0.52 () {60 vars in 1482 bytes} [Wed Jul 13 22:14:50 2022] PUT /identity/v3/projects/ec4d654647ad4d35ba6475136226be74/users/97f3be025adf4ffeacdfcb212e72e168/roles/2a5e86976394498abb6310697cdb3f8b => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 13 22:14:50.572857 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c41651-6b10-4cdd-92c3-710273dc3187 None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:14:50.573143 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c41651-6b10-4cdd-92c3-710273dc3187 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:14:50.573443 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-59c41651-6b10-4cdd-92c3-710273dc3187 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:14:50.601272 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-59c41651-6b10-4cdd-92c3-710273dc3187 None None] MFA Rules not processed for user `97f3be025adf4ffeacdfcb212e72e168`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 22:14:50.602421 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-59c41651-6b10-4cdd-92c3-710273dc3187 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:14:50.673116 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1103/2212] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jul 13 22:14:50 2022] POST /identity/v3/auth/tokens => generated 3232 bytes in 102 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 22:14:50.677773 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7b0046-f7b8-42d3-a3b6-4e35410aff6b admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:14:50.741112 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc7b0046-f7b8-42d3-a3b6-4e35410aff6b admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:14:50.746073 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314dd8af-ac50-4350-bf44-edabccfa954c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:14:50.746759 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314dd8af-ac50-4350-bf44-edabccfa954c admin admin] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:14:50.747034 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314dd8af-ac50-4350-bf44-edabccfa954c admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:14:50.747301 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-314dd8af-ac50-4350-bf44-edabccfa954c admin admin] PATH_INFO: `/v3/projects` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:14:50.747987 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314dd8af-ac50-4350-bf44-edabccfa954c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:14:50.749931 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-314dd8af-ac50-4350-bf44-edabccfa954c admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:14:50.778220 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1110/2213] 10.222.0.52 () {60 vars in 1152 bytes} [Wed Jul 13 22:14:50 2022] POST /identity/v3/projects => generated 367 bytes in 101 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 22:14:50.782412 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c41651-6b10-4cdd-92c3-710273dc3187 None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:14:50.793903 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-59c41651-6b10-4cdd-92c3-710273dc3187 None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:14:50.854105 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692bdf29-2181-41f7-988a-f31d2abe3bc4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:14:50.854734 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692bdf29-2181-41f7-988a-f31d2abe3bc4 admin admin] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:14:50.855019 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692bdf29-2181-41f7-988a-f31d2abe3bc4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:14:50.855273 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-692bdf29-2181-41f7-988a-f31d2abe3bc4 admin admin] PATH_INFO: `/v3/users` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:14:50.856001 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692bdf29-2181-41f7-988a-f31d2abe3bc4 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:14:50.857934 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-692bdf29-2181-41f7-988a-f31d2abe3bc4 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:14:50.879085 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1104/2214] 10.222.0.52 () {60 vars in 1143 bytes} [Wed Jul 13 22:14:50 2022] POST /identity/v3/users => generated 290 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 22:14:50.883343 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314dd8af-ac50-4350-bf44-edabccfa954c admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:14:50.894445 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-314dd8af-ac50-4350-bf44-edabccfa954c admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:14:50.899788 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a138df-b40c-47c3-aa5e-ccff7d6471ad admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:14:50.900602 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a138df-b40c-47c3-aa5e-ccff7d6471ad admin admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:14:50.900889 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a138df-b40c-47c3-aa5e-ccff7d6471ad admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:14:50.901169 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-24a138df-b40c-47c3-aa5e-ccff7d6471ad admin admin] PATH_INFO: `/v3/roles` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:14:50.901935 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a138df-b40c-47c3-aa5e-ccff7d6471ad admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:14:50.903600 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-24a138df-b40c-47c3-aa5e-ccff7d6471ad admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:14:50.914851 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1111/2215] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jul 13 22:14:50 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 22:14:50.919227 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692bdf29-2181-41f7-988a-f31d2abe3bc4 admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:14:50.930113 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-692bdf29-2181-41f7-988a-f31d2abe3bc4 admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:14:50.935458 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d09df1d-fb06-409b-8b90-1491b9585af3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:14:50.936409 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d09df1d-fb06-409b-8b90-1491b9585af3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:14:50.936683 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d09df1d-fb06-409b-8b90-1491b9585af3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:14:50.936956 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d09df1d-fb06-409b-8b90-1491b9585af3 admin admin] PATH_INFO: `/v3/projects/d772de4a538245708ab6f25792749fa1/users/b64cb4bfe1564e3ba2c24f0338a5f52c/roles/44efba2579f342df8a0789c59d163ec3` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:14:50.954976 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d09df1d-fb06-409b-8b90-1491b9585af3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d772de4a538245708ab6f25792749fa1, user_id=b64cb4bfe1564e3ba2c24f0338a5f52c, role_id=44efba2579f342df8a0789c59d163ec3)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:14:50.958500 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d09df1d-fb06-409b-8b90-1491b9585af3 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:14:50.974098 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1105/2216] 10.222.0.52 () {60 vars in 1482 bytes} [Wed Jul 13 22:14:50 2022] PUT /identity/v3/projects/d772de4a538245708ab6f25792749fa1/users/b64cb4bfe1564e3ba2c24f0338a5f52c/roles/44efba2579f342df8a0789c59d163ec3 => generated 0 bytes in 55 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 13 22:14:50.978950 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a138df-b40c-47c3-aa5e-ccff7d6471ad admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:14:50.989310 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-24a138df-b40c-47c3-aa5e-ccff7d6471ad admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:14:51.060978 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a7e9e8cf-18ee-4d17-8e5c-bc4c3e9ebe99 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:14:51.061658 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e9e8cf-18ee-4d17-8e5c-bc4c3e9ebe99 admin admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:14:51.061941 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e9e8cf-18ee-4d17-8e5c-bc4c3e9ebe99 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:14:51.062215 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a7e9e8cf-18ee-4d17-8e5c-bc4c3e9ebe99 admin admin] PATH_INFO: `/v3/roles` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:14:51.063006 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e9e8cf-18ee-4d17-8e5c-bc4c3e9ebe99 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:14:51.064656 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a7e9e8cf-18ee-4d17-8e5c-bc4c3e9ebe99 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:14:51.075143 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1112/2217] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jul 13 22:14:50 2022] GET /identity/v3/roles => generated 1420 bytes in 97 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 22:14:51.079513 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d09df1d-fb06-409b-8b90-1491b9585af3 admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:14:51.090380 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d09df1d-fb06-409b-8b90-1491b9585af3 admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:14:51.095654 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36feab1e-44f6-4acc-907d-2abba39e8d16 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:14:51.096538 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36feab1e-44f6-4acc-907d-2abba39e8d16 admin admin] REQUEST_METHOD: `PUT` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:14:51.096817 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36feab1e-44f6-4acc-907d-2abba39e8d16 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:14:51.097056 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-36feab1e-44f6-4acc-907d-2abba39e8d16 admin admin] PATH_INFO: `/v3/projects/d772de4a538245708ab6f25792749fa1/users/b64cb4bfe1564e3ba2c24f0338a5f52c/roles/44efba2579f342df8a0789c59d163ec3` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:14:51.099390 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36feab1e-44f6-4acc-907d-2abba39e8d16 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d772de4a538245708ab6f25792749fa1, user_id=b64cb4bfe1564e3ba2c24f0338a5f52c, role_id=44efba2579f342df8a0789c59d163ec3)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:14:51.102463 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-36feab1e-44f6-4acc-907d-2abba39e8d16 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:14:51.108902 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=92,keystone:INSERT=8 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:14:51.114889 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1106/2218] 10.222.0.52 () {60 vars in 1482 bytes} [Wed Jul 13 22:14:51 2022] PUT /identity/v3/projects/d772de4a538245708ab6f25792749fa1/users/b64cb4bfe1564e3ba2c24f0338a5f52c/roles/44efba2579f342df8a0789c59d163ec3 => generated 0 bytes in 36 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 22:14:51.183618 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf59ca67-5526-4eee-b5f2-8dd15989abe2 None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:14:51.183900 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf59ca67-5526-4eee-b5f2-8dd15989abe2 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:14:51.184174 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bf59ca67-5526-4eee-b5f2-8dd15989abe2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:14:51.216016 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-bf59ca67-5526-4eee-b5f2-8dd15989abe2 None None] MFA Rules not processed for user `b64cb4bfe1564e3ba2c24f0338a5f52c`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 22:14:51.217224 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-bf59ca67-5526-4eee-b5f2-8dd15989abe2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:14:51.281004 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:14:51.293273 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1113/2219] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jul 13 22:14:51 2022] POST /identity/v3/auth/tokens => generated 3301 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 22:14:51.304991 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36feab1e-44f6-4acc-907d-2abba39e8d16 admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:14:51.306826 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-36feab1e-44f6-4acc-907d-2abba39e8d16 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:14:51.370823 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-36feab1e-44f6-4acc-907d-2abba39e8d16 admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:14:51.375078 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6395107-754c-4b6b-85f0-331bb8ef29fd service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc77e1ea1ada4700b809beb1c980c5e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417e3e2f1333481795a089bef2e24bea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:14:51.375726 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6395107-754c-4b6b-85f0-331bb8ef29fd service cinder] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:14:51.376004 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6395107-754c-4b6b-85f0-331bb8ef29fd service cinder] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:14:51.376263 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a6395107-754c-4b6b-85f0-331bb8ef29fd service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:14:51.386006 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6395107-754c-4b6b-85f0-331bb8ef29fd service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:14:51.387481 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a6395107-754c-4b6b-85f0-331bb8ef29fd service cinder] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:14:51.397328 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1107/2220] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jul 13 22:14:51 2022] GET /identity/v3/auth/tokens => generated 3301 bytes in 93 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 22:14:51.479624 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf59ca67-5526-4eee-b5f2-8dd15989abe2 None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:14:51.490128 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bf59ca67-5526-4eee-b5f2-8dd15989abe2 None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:14:51.494423 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-874f75b2-446b-4cc2-bed9-121763e0d15b service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc77e1ea1ada4700b809beb1c980c5e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417e3e2f1333481795a089bef2e24bea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:14:51.495220 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874f75b2-446b-4cc2-bed9-121763e0d15b service cinder] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:14:51.495507 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874f75b2-446b-4cc2-bed9-121763e0d15b service cinder] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:14:51.495773 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-874f75b2-446b-4cc2-bed9-121763e0d15b service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:14:51.570896 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874f75b2-446b-4cc2-bed9-121763e0d15b service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:14:51.570896 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-874f75b2-446b-4cc2-bed9-121763e0d15b service cinder] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:14:51.579643 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1114/2221] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jul 13 22:14:51 2022] GET /identity/v3/auth/tokens => generated 3232 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 22:14:57.580351 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6395107-754c-4b6b-85f0-331bb8ef29fd service cinder] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:14:57.649436 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a6395107-754c-4b6b-85f0-331bb8ef29fd service cinder] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:14:57.655395 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f763dba5-870a-411a-87e2-75c8e5f08c3b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:14:57.656498 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f763dba5-870a-411a-87e2-75c8e5f08c3b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:14:57.656816 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f763dba5-870a-411a-87e2-75c8e5f08c3b admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:14:57.657092 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f763dba5-870a-411a-87e2-75c8e5f08c3b admin admin] PATH_INFO: `/v3/users/97f3be025adf4ffeacdfcb212e72e168` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:14:57.658392 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f763dba5-870a-411a-87e2-75c8e5f08c3b admin admin] RBAC: Authorizing `identity:delete_user(user_id=97f3be025adf4ffeacdfcb212e72e168)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:14:57.660451 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f763dba5-870a-411a-87e2-75c8e5f08c3b admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:14:57.715461 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-f763dba5-870a-411a-87e2-75c8e5f08c3b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '97f3be025adf4ffeacdfcb212e72e168'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:14:57.724495 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-f763dba5-870a-411a-87e2-75c8e5f08c3b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '97f3be025adf4ffeacdfcb212e72e168'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:14:57.737305 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-f763dba5-870a-411a-87e2-75c8e5f08c3b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '97f3be025adf4ffeacdfcb212e72e168'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:14:57.737996 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-f763dba5-870a-411a-87e2-75c8e5f08c3b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '97f3be025adf4ffeacdfcb212e72e168'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:14:57.738632 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-f763dba5-870a-411a-87e2-75c8e5f08c3b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '97f3be025adf4ffeacdfcb212e72e168'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:14:57.751787 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1108/2222] 10.222.0.52 () {58 vars in 1223 bytes} [Wed Jul 13 22:14:57 2022] DELETE /identity/v3/users/97f3be025adf4ffeacdfcb212e72e168 => generated 0 bytes in 172 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 22:14:57.756267 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cf54d3-5263-490b-8e24-ee76f0c94549 None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:14:57.756806 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cf54d3-5263-490b-8e24-ee76f0c94549 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:14:57.756806 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-25cf54d3-5263-490b-8e24-ee76f0c94549 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:14:57.780326 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-25cf54d3-5263-490b-8e24-ee76f0c94549 None None] MFA Rules not processed for user `cc99c93da5074cf5a48bc2aa9a92c5e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 22:14:57.781571 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-25cf54d3-5263-490b-8e24-ee76f0c94549 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:14:57.848659 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1115/2223] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jul 13 22:14:57 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 93 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 22:14:57.858553 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f763dba5-870a-411a-87e2-75c8e5f08c3b admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:14:57.860203 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-f763dba5-870a-411a-87e2-75c8e5f08c3b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:14:57.883332 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:14:57.923620 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f763dba5-870a-411a-87e2-75c8e5f08c3b admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:14:57.927701 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df95808-1441-463e-9d1a-b68ced4e1c0f service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '847ec9a1bd4d4dec804318a5c845895a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417e3e2f1333481795a089bef2e24bea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:14:57.928353 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df95808-1441-463e-9d1a-b68ced4e1c0f service neutron] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:14:57.928634 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df95808-1441-463e-9d1a-b68ced4e1c0f service neutron] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:14:57.928903 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2df95808-1441-463e-9d1a-b68ced4e1c0f service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:14:57.938159 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df95808-1441-463e-9d1a-b68ced4e1c0f service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:14:57.939628 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2df95808-1441-463e-9d1a-b68ced4e1c0f service neutron] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:14:57.949484 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1109/2224] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jul 13 22:14:57 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 22:14:58.236629 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25cf54d3-5263-490b-8e24-ee76f0c94549 None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:14:58.238206 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-25cf54d3-5263-490b-8e24-ee76f0c94549 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:14:58.252598 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-25cf54d3-5263-490b-8e24-ee76f0c94549 None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:14:58.258803 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffbce8e-ade5-4bf8-b840-cdf0b3c5cf4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:14:58.259979 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffbce8e-ade5-4bf8-b840-cdf0b3c5cf4c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:14:58.260486 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffbce8e-ade5-4bf8-b840-cdf0b3c5cf4c admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:14:58.260827 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dffbce8e-ade5-4bf8-b840-cdf0b3c5cf4c admin admin] PATH_INFO: `/v3/projects/ec4d654647ad4d35ba6475136226be74` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:14:58.262163 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dffbce8e-ade5-4bf8-b840-cdf0b3c5cf4c admin admin] RBAC: Authorizing `identity:delete_project(project_id=ec4d654647ad4d35ba6475136226be74)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:14:58.264278 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dffbce8e-ade5-4bf8-b840-cdf0b3c5cf4c admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:14:58.315076 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-dffbce8e-ade5-4bf8-b840-cdf0b3c5cf4c admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ec4d654647ad4d35ba6475136226be74', 'request_id': 'req-dffbce8e-ade5-4bf8-b840-cdf0b3c5cf4c'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:14:58.322273 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-dffbce8e-ade5-4bf8-b840-cdf0b3c5cf4c admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ec4d654647ad4d35ba6475136226be74', 'request_id': 'req-dffbce8e-ade5-4bf8-b840-cdf0b3c5cf4c'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:14:58.330751 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-dffbce8e-ade5-4bf8-b840-cdf0b3c5cf4c admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ec4d654647ad4d35ba6475136226be74', 'request_id': 'req-dffbce8e-ade5-4bf8-b840-cdf0b3c5cf4c'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:14:58.345437 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-dffbce8e-ade5-4bf8-b840-cdf0b3c5cf4c admin admin] The token cache is being invalidate because project ec4d654647ad4d35ba6475136226be74 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80381) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 22:14:58.345728 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-dffbce8e-ade5-4bf8-b840-cdf0b3c5cf4c admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:14:58.346340 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-dffbce8e-ade5-4bf8-b840-cdf0b3c5cf4c admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:14:58.347927 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1116/2225] 10.222.0.52 () {58 vars in 1232 bytes} [Wed Jul 13 22:14:58 2022] DELETE /identity/v3/projects/ec4d654647ad4d35ba6475136226be74 => generated 0 bytes in 112 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 22:14:58.352372 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df95808-1441-463e-9d1a-b68ced4e1c0f service neutron] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:14:58.362511 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2df95808-1441-463e-9d1a-b68ced4e1c0f service neutron] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:14:58.422665 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f49ceaa-f2f6-4846-ae2a-176ae5819a94 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:14:58.423077 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f49ceaa-f2f6-4846-ae2a-176ae5819a94 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:14:58.423357 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f49ceaa-f2f6-4846-ae2a-176ae5819a94 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:14:58.423631 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f49ceaa-f2f6-4846-ae2a-176ae5819a94 admin admin] PATH_INFO: `/v3/users/b64cb4bfe1564e3ba2c24f0338a5f52c` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:14:58.424822 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f49ceaa-f2f6-4846-ae2a-176ae5819a94 admin admin] RBAC: Authorizing `identity:delete_user(user_id=b64cb4bfe1564e3ba2c24f0338a5f52c)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:14:58.426779 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f49ceaa-f2f6-4846-ae2a-176ae5819a94 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:14:58.477668 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-3f49ceaa-f2f6-4846-ae2a-176ae5819a94 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'b64cb4bfe1564e3ba2c24f0338a5f52c'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:14:58.487169 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-3f49ceaa-f2f6-4846-ae2a-176ae5819a94 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'b64cb4bfe1564e3ba2c24f0338a5f52c'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:14:58.499442 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-3f49ceaa-f2f6-4846-ae2a-176ae5819a94 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'b64cb4bfe1564e3ba2c24f0338a5f52c'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:14:58.500018 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-3f49ceaa-f2f6-4846-ae2a-176ae5819a94 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'b64cb4bfe1564e3ba2c24f0338a5f52c'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:14:58.500625 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-3f49ceaa-f2f6-4846-ae2a-176ae5819a94 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'b64cb4bfe1564e3ba2c24f0338a5f52c'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:14:58.512931 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1110/2226] 10.222.0.52 () {58 vars in 1223 bytes} [Wed Jul 13 22:14:58 2022] DELETE /identity/v3/users/b64cb4bfe1564e3ba2c24f0338a5f52c => generated 0 bytes in 161 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 22:14:58.871925 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffbce8e-ade5-4bf8-b840-cdf0b3c5cf4c admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:14:58.873407 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-dffbce8e-ade5-4bf8-b840-cdf0b3c5cf4c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:14:58.886676 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dffbce8e-ade5-4bf8-b840-cdf0b3c5cf4c admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:14:58.905177 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:14:58.955003 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a2bbab53-e533-4211-9926-765d1ff0a531 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:14:58.955890 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2bbab53-e533-4211-9926-765d1ff0a531 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:14:58.956168 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2bbab53-e533-4211-9926-765d1ff0a531 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:14:58.956439 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a2bbab53-e533-4211-9926-765d1ff0a531 admin admin] PATH_INFO: `/v3/projects/d772de4a538245708ab6f25792749fa1` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:14:58.957640 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2bbab53-e533-4211-9926-765d1ff0a531 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d772de4a538245708ab6f25792749fa1)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:14:58.959606 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a2bbab53-e533-4211-9926-765d1ff0a531 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:14:59.006167 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-a2bbab53-e533-4211-9926-765d1ff0a531 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd772de4a538245708ab6f25792749fa1', 'request_id': 'req-a2bbab53-e533-4211-9926-765d1ff0a531'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:14:59.012391 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-a2bbab53-e533-4211-9926-765d1ff0a531 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd772de4a538245708ab6f25792749fa1', 'request_id': 'req-a2bbab53-e533-4211-9926-765d1ff0a531'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:14:59.018610 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-a2bbab53-e533-4211-9926-765d1ff0a531 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd772de4a538245708ab6f25792749fa1', 'request_id': 'req-a2bbab53-e533-4211-9926-765d1ff0a531'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:14:59.033984 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-a2bbab53-e533-4211-9926-765d1ff0a531 admin admin] The token cache is being invalidate because project d772de4a538245708ab6f25792749fa1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80381) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 22:14:59.034259 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-a2bbab53-e533-4211-9926-765d1ff0a531 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:14:59.035014 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-a2bbab53-e533-4211-9926-765d1ff0a531 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:14:59.036718 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1117/2227] 10.222.0.52 () {58 vars in 1232 bytes} [Wed Jul 13 22:14:58 2022] DELETE /identity/v3/projects/d772de4a538245708ab6f25792749fa1 => generated 0 bytes in 166 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 22:14:59.041913 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03bd119-4e7c-4a53-b422-a07e2c65d12a None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:14:59.042706 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03bd119-4e7c-4a53-b422-a07e2c65d12a None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:14:59.042706 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c03bd119-4e7c-4a53-b422-a07e2c65d12a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:14:59.063656 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-c03bd119-4e7c-4a53-b422-a07e2c65d12a None None] MFA Rules not processed for user `cc99c93da5074cf5a48bc2aa9a92c5e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 22:14:59.064815 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-c03bd119-4e7c-4a53-b422-a07e2c65d12a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:14:59.105386 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:14:59.128426 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1111/2228] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jul 13 22:14:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 88 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 22:14:59.136809 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bab66b6-220e-4a03-8ad5-a7bc2001efad None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:14:59.136809 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bab66b6-220e-4a03-8ad5-a7bc2001efad None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:14:59.137098 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bab66b6-220e-4a03-8ad5-a7bc2001efad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:14:59.157344 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-1bab66b6-220e-4a03-8ad5-a7bc2001efad None None] MFA Rules not processed for user `cc99c93da5074cf5a48bc2aa9a92c5e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 22:14:59.158631 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-1bab66b6-220e-4a03-8ad5-a7bc2001efad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:14:59.167709 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1118/2229] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jul 13 22:14:59 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 22:14:59.172780 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03bd119-4e7c-4a53-b422-a07e2c65d12a None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:14:59.185275 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c03bd119-4e7c-4a53-b422-a07e2c65d12a None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:14:59.190296 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa02538-a604-4ca5-a0cd-cc7b98d0bcf2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:14:59.190998 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa02538-a604-4ca5-a0cd-cc7b98d0bcf2 admin admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:14:59.191279 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa02538-a604-4ca5-a0cd-cc7b98d0bcf2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:14:59.191576 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9fa02538-a604-4ca5-a0cd-cc7b98d0bcf2 admin admin] PATH_INFO: `/v3/domains` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:14:59.192317 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa02538-a604-4ca5-a0cd-cc7b98d0bcf2 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 22:14:59.192649 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa02538-a604-4ca5-a0cd-cc7b98d0bcf2 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:14:59.194264 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9fa02538-a604-4ca5-a0cd-cc7b98d0bcf2 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:14:59.206936 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1112/2230] 10.222.0.52 () {58 vars in 1152 bytes} [Wed Jul 13 22:14:59 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 22:14:59.211479 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bab66b6-220e-4a03-8ad5-a7bc2001efad None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:14:59.221124 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bab66b6-220e-4a03-8ad5-a7bc2001efad None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:14:59.227360 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38beebe-743d-49f4-bfa8-5ec3f717989e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:14:59.228026 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38beebe-743d-49f4-bfa8-5ec3f717989e admin admin] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:14:59.228308 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38beebe-743d-49f4-bfa8-5ec3f717989e admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:14:59.228582 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d38beebe-743d-49f4-bfa8-5ec3f717989e admin admin] PATH_INFO: `/v3/projects` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:14:59.229332 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38beebe-743d-49f4-bfa8-5ec3f717989e admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:14:59.231343 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d38beebe-743d-49f4-bfa8-5ec3f717989e admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:14:59.261245 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1119/2231] 10.222.0.52 () {60 vars in 1152 bytes} [Wed Jul 13 22:14:59 2022] POST /identity/v3/projects => generated 375 bytes in 50 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 22:14:59.265265 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa02538-a604-4ca5-a0cd-cc7b98d0bcf2 admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:14:59.275480 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9fa02538-a604-4ca5-a0cd-cc7b98d0bcf2 admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:14:59.333766 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289ff7ca-aa8c-4bd4-9668-bd054e690fe9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:14:59.334370 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289ff7ca-aa8c-4bd4-9668-bd054e690fe9 admin admin] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:14:59.334692 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289ff7ca-aa8c-4bd4-9668-bd054e690fe9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:14:59.335060 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-289ff7ca-aa8c-4bd4-9668-bd054e690fe9 admin admin] PATH_INFO: `/v3/users` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:14:59.335765 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289ff7ca-aa8c-4bd4-9668-bd054e690fe9 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:14:59.337732 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-289ff7ca-aa8c-4bd4-9668-bd054e690fe9 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:14:59.358933 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1113/2232] 10.222.0.52 () {60 vars in 1143 bytes} [Wed Jul 13 22:14:59 2022] POST /identity/v3/users => generated 288 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 22:14:59.363205 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38beebe-743d-49f4-bfa8-5ec3f717989e admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:14:59.375909 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d38beebe-743d-49f4-bfa8-5ec3f717989e admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:14:59.380951 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a37816b-af35-40ff-b5e0-21932608b167 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:14:59.381633 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a37816b-af35-40ff-b5e0-21932608b167 admin admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:14:59.381917 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a37816b-af35-40ff-b5e0-21932608b167 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:14:59.382182 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a37816b-af35-40ff-b5e0-21932608b167 admin admin] PATH_INFO: `/v3/roles` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:14:59.382969 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a37816b-af35-40ff-b5e0-21932608b167 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:14:59.384651 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a37816b-af35-40ff-b5e0-21932608b167 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:14:59.397403 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1120/2233] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jul 13 22:14:59 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 22:14:59.401689 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289ff7ca-aa8c-4bd4-9668-bd054e690fe9 admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:14:59.413354 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-289ff7ca-aa8c-4bd4-9668-bd054e690fe9 admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:14:59.418993 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bbdb8ac-066d-4987-9f02-4296a9f0bfd7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:14:59.419666 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbdb8ac-066d-4987-9f02-4296a9f0bfd7 admin admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:14:59.419970 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbdb8ac-066d-4987-9f02-4296a9f0bfd7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:14:59.420245 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bbdb8ac-066d-4987-9f02-4296a9f0bfd7 admin admin] PATH_INFO: `/v3/roles` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:14:59.421037 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bbdb8ac-066d-4987-9f02-4296a9f0bfd7 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:14:59.422709 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bbdb8ac-066d-4987-9f02-4296a9f0bfd7 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:14:59.433809 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1114/2234] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jul 13 22:14:59 2022] GET /identity/v3/roles => generated 1420 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 22:14:59.438106 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a37816b-af35-40ff-b5e0-21932608b167 admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:14:59.449180 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a37816b-af35-40ff-b5e0-21932608b167 admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:14:59.453829 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87bd0f0-346d-401f-a10b-a8af31896245 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:14:59.454759 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87bd0f0-346d-401f-a10b-a8af31896245 admin admin] REQUEST_METHOD: `PUT` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:14:59.455038 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87bd0f0-346d-401f-a10b-a8af31896245 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:14:59.455309 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e87bd0f0-346d-401f-a10b-a8af31896245 admin admin] PATH_INFO: `/v3/projects/b59379c9d3b34c51b4af356ae41eb3fe/users/619af5dee82a4342a1bca6aee37b9b69/roles/2a5e86976394498abb6310697cdb3f8b` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:14:59.478674 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e87bd0f0-346d-401f-a10b-a8af31896245 admin admin] RBAC: Authorizing `identity:create_grant(project_id=b59379c9d3b34c51b4af356ae41eb3fe, user_id=619af5dee82a4342a1bca6aee37b9b69, role_id=2a5e86976394498abb6310697cdb3f8b)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:14:59.483951 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e87bd0f0-346d-401f-a10b-a8af31896245 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:14:59.498914 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1121/2235] 10.222.0.52 () {60 vars in 1482 bytes} [Wed Jul 13 22:14:59 2022] PUT /identity/v3/projects/b59379c9d3b34c51b4af356ae41eb3fe/users/619af5dee82a4342a1bca6aee37b9b69/roles/2a5e86976394498abb6310697cdb3f8b => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 22:14:59.510688 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a48421c-8dca-4d7e-9210-37b4ba1a7a6b None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:14:59.510978 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a48421c-8dca-4d7e-9210-37b4ba1a7a6b None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:14:59.511382 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2a48421c-8dca-4d7e-9210-37b4ba1a7a6b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:14:59.544768 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-2a48421c-8dca-4d7e-9210-37b4ba1a7a6b None None] MFA Rules not processed for user `619af5dee82a4342a1bca6aee37b9b69`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 22:14:59.545979 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-2a48421c-8dca-4d7e-9210-37b4ba1a7a6b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:14:59.615342 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1115/2236] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jul 13 22:14:59 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 22:14:59.627630 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87bd0f0-346d-401f-a10b-a8af31896245 admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:14:59.629583 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-e87bd0f0-346d-401f-a10b-a8af31896245 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:14:59.688012 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=90,keystone:DELETE=7,keystone:INSERT=3 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:14:59.702672 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e87bd0f0-346d-401f-a10b-a8af31896245 admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:14:59.706564 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1709134-746e-4143-829a-9b8ded17ee05 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc77e1ea1ada4700b809beb1c980c5e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417e3e2f1333481795a089bef2e24bea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:14:59.707301 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1709134-746e-4143-829a-9b8ded17ee05 service cinder] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:14:59.707580 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1709134-746e-4143-829a-9b8ded17ee05 service cinder] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:14:59.707851 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1709134-746e-4143-829a-9b8ded17ee05 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:14:59.716717 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1709134-746e-4143-829a-9b8ded17ee05 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:14:59.718171 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1709134-746e-4143-829a-9b8ded17ee05 service cinder] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:14:59.727332 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1122/2237] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jul 13 22:14:59 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 22:15:01.156580 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add9420b-4cbc-40d7-a7a0-5d81579b088c None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:15:01.157095 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add9420b-4cbc-40d7-a7a0-5d81579b088c None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:15:01.157196 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-add9420b-4cbc-40d7-a7a0-5d81579b088c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:15:01.175119 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-add9420b-4cbc-40d7-a7a0-5d81579b088c None None] MFA Rules not processed for user `619af5dee82a4342a1bca6aee37b9b69`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 22:15:01.176337 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-add9420b-4cbc-40d7-a7a0-5d81579b088c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:15:01.187743 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1116/2238] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jul 13 22:15:01 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 32 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 22:15:04.429949 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1709134-746e-4143-829a-9b8ded17ee05 service cinder] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:15:04.431620 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-d1709134-746e-4143-829a-9b8ded17ee05 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:15:04.500581 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1709134-746e-4143-829a-9b8ded17ee05 service cinder] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:15:04.505654 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eebb91e1-eac9-4d70-bf29-67160ca8ac43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:15:04.506612 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebb91e1-eac9-4d70-bf29-67160ca8ac43 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:15:04.506886 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebb91e1-eac9-4d70-bf29-67160ca8ac43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:15:04.507158 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-eebb91e1-eac9-4d70-bf29-67160ca8ac43 admin admin] PATH_INFO: `/v3/users/96b12071c9bd45baba003ec6253136a1` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:15:04.508401 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eebb91e1-eac9-4d70-bf29-67160ca8ac43 admin admin] RBAC: Authorizing `identity:delete_user(user_id=96b12071c9bd45baba003ec6253136a1)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:15:04.510315 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-eebb91e1-eac9-4d70-bf29-67160ca8ac43 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:15:04.572815 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-eebb91e1-eac9-4d70-bf29-67160ca8ac43 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '96b12071c9bd45baba003ec6253136a1'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:15:04.581501 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-eebb91e1-eac9-4d70-bf29-67160ca8ac43 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '96b12071c9bd45baba003ec6253136a1'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:15:04.593937 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-eebb91e1-eac9-4d70-bf29-67160ca8ac43 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '96b12071c9bd45baba003ec6253136a1'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:15:04.594783 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-eebb91e1-eac9-4d70-bf29-67160ca8ac43 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '96b12071c9bd45baba003ec6253136a1'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:15:04.599842 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-eebb91e1-eac9-4d70-bf29-67160ca8ac43 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '96b12071c9bd45baba003ec6253136a1'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:15:04.612491 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1123/2239] 10.222.0.52 () {58 vars in 1223 bytes} [Wed Jul 13 22:15:04 2022] DELETE /identity/v3/users/96b12071c9bd45baba003ec6253136a1 => generated 0 bytes in 183 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 22:15:04.617118 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79b9c21-c8c6-4ace-bfbc-34de8f6e9ba2 None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:15:04.617622 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79b9c21-c8c6-4ace-bfbc-34de8f6e9ba2 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:15:04.617739 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d79b9c21-c8c6-4ace-bfbc-34de8f6e9ba2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:15:04.636569 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-d79b9c21-c8c6-4ace-bfbc-34de8f6e9ba2 None None] MFA Rules not processed for user `cc99c93da5074cf5a48bc2aa9a92c5e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 22:15:04.637756 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-d79b9c21-c8c6-4ace-bfbc-34de8f6e9ba2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:15:04.665176 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:15:04.701151 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1117/2240] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jul 13 22:15:04 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 22:15:04.711687 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eebb91e1-eac9-4d70-bf29-67160ca8ac43 admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:15:04.714029 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-eebb91e1-eac9-4d70-bf29-67160ca8ac43 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:15:04.786615 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-eebb91e1-eac9-4d70-bf29-67160ca8ac43 admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:15:04.791062 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5e4eac07-6fb0-4e74-8d17-4c89964308d3 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '847ec9a1bd4d4dec804318a5c845895a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417e3e2f1333481795a089bef2e24bea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:15:04.791710 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4eac07-6fb0-4e74-8d17-4c89964308d3 service neutron] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:15:04.791988 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4eac07-6fb0-4e74-8d17-4c89964308d3 service neutron] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:15:04.792259 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5e4eac07-6fb0-4e74-8d17-4c89964308d3 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:15:04.802466 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4eac07-6fb0-4e74-8d17-4c89964308d3 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:15:04.803928 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5e4eac07-6fb0-4e74-8d17-4c89964308d3 service neutron] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:15:04.813664 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1124/2241] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jul 13 22:15:04 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 22:15:05.134466 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79b9c21-c8c6-4ace-bfbc-34de8f6e9ba2 None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:15:05.136418 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-d79b9c21-c8c6-4ace-bfbc-34de8f6e9ba2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:15:05.150922 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d79b9c21-c8c6-4ace-bfbc-34de8f6e9ba2 None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:15:05.156811 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f15366-8328-4695-82fa-c495640fe192 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:15:05.157917 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f15366-8328-4695-82fa-c495640fe192 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:15:05.158282 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f15366-8328-4695-82fa-c495640fe192 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:15:05.159227 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-26f15366-8328-4695-82fa-c495640fe192 admin admin] PATH_INFO: `/v3/projects/3d82e209d4504d6d81e3b54444fb59ed` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:15:05.160702 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f15366-8328-4695-82fa-c495640fe192 admin admin] RBAC: Authorizing `identity:delete_project(project_id=3d82e209d4504d6d81e3b54444fb59ed)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:15:05.162705 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-26f15366-8328-4695-82fa-c495640fe192 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:15:05.217717 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-26f15366-8328-4695-82fa-c495640fe192 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '3d82e209d4504d6d81e3b54444fb59ed', 'request_id': 'req-26f15366-8328-4695-82fa-c495640fe192'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:15:05.230420 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-26f15366-8328-4695-82fa-c495640fe192 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '3d82e209d4504d6d81e3b54444fb59ed', 'request_id': 'req-26f15366-8328-4695-82fa-c495640fe192'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:15:05.239101 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-26f15366-8328-4695-82fa-c495640fe192 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '3d82e209d4504d6d81e3b54444fb59ed', 'request_id': 'req-26f15366-8328-4695-82fa-c495640fe192'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:15:05.257363 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-26f15366-8328-4695-82fa-c495640fe192 admin admin] The token cache is being invalidate because project 3d82e209d4504d6d81e3b54444fb59ed was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80382) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 22:15:05.257650 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-26f15366-8328-4695-82fa-c495640fe192 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:15:05.258268 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-26f15366-8328-4695-82fa-c495640fe192 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:15:05.260113 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1118/2242] 10.222.0.52 () {58 vars in 1232 bytes} [Wed Jul 13 22:15:05 2022] DELETE /identity/v3/projects/3d82e209d4504d6d81e3b54444fb59ed => generated 0 bytes in 127 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 22:15:08.637461 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acaf92c6-265c-461e-9d66-82012f801101 None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:15:08.638007 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acaf92c6-265c-461e-9d66-82012f801101 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:15:08.638007 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-acaf92c6-265c-461e-9d66-82012f801101 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:15:08.651132 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:15:08.659606 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-acaf92c6-265c-461e-9d66-82012f801101 None None] MFA Rules not processed for user `45b5d8b0ba324f629f697cf0de5918bb`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 22:15:08.661348 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-acaf92c6-265c-461e-9d66-82012f801101 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:15:08.723442 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1125/2243] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jul 13 22:15:08 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 87 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 22:15:15.254305 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=40,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:15:18.716135 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=22 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:15:19.903512 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f15366-8328-4695-82fa-c495640fe192 admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:15:19.906930 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-26f15366-8328-4695-82fa-c495640fe192 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:15:20.000794 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-26f15366-8328-4695-82fa-c495640fe192 admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:15:20.006305 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d85f83-4a93-4448-ad7f-ec278ae1f0d0 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac673d641d8849c79bf4cf71c9805587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417e3e2f1333481795a089bef2e24bea', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:15:20.007015 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d85f83-4a93-4448-ad7f-ec278ae1f0d0 service placement] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:15:20.007292 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d85f83-4a93-4448-ad7f-ec278ae1f0d0 service placement] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:15:20.007565 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9d85f83-4a93-4448-ad7f-ec278ae1f0d0 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:15:20.009457 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-b9d85f83-4a93-4448-ad7f-ec278ae1f0d0 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:15:20.021600 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d85f83-4a93-4448-ad7f-ec278ae1f0d0 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:15:20.023238 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9d85f83-4a93-4448-ad7f-ec278ae1f0d0 service placement] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:15:20.033281 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1119/2244] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jul 13 22:15:19 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 22:15:30.029281 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=32 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:15:40.523210 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b3396a-7813-4d29-98f2-61c7a6f8491b None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:15:40.523735 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b3396a-7813-4d29-98f2-61c7a6f8491b None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:15:40.523735 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3b3396a-7813-4d29-98f2-61c7a6f8491b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:15:40.541810 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-c3b3396a-7813-4d29-98f2-61c7a6f8491b None None] MFA Rules not processed for user `b866bacdafae4db6a32714c07c56515f`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 22:15:40.543155 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-c3b3396a-7813-4d29-98f2-61c7a6f8491b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:15:40.601315 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1126/2245] 10.222.0.52 () {58 vars in 1009 bytes} [Wed Jul 13 22:15:40 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 79 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 22:15:40.641288 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d85f83-4a93-4448-ad7f-ec278ae1f0d0 service placement] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:15:40.642907 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-b9d85f83-4a93-4448-ad7f-ec278ae1f0d0 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:15:40.715585 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9d85f83-4a93-4448-ad7f-ec278ae1f0d0 service placement] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:15:40.720983 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bddce29-ec7c-4c68-b5ba-a0fc14d0466a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eb1525233ba48c99b15d4cb75db4822', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417e3e2f1333481795a089bef2e24bea', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:15:40.721682 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bddce29-ec7c-4c68-b5ba-a0fc14d0466a service swift] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:15:40.721966 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bddce29-ec7c-4c68-b5ba-a0fc14d0466a service swift] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:15:40.722241 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7bddce29-ec7c-4c68-b5ba-a0fc14d0466a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:15:40.731397 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bddce29-ec7c-4c68-b5ba-a0fc14d0466a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:15:40.732985 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7bddce29-ec7c-4c68-b5ba-a0fc14d0466a service swift] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:15:40.740933 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1120/2246] 10.222.0.52 () {60 vars in 1384 bytes} [Wed Jul 13 22:15:40 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 13 22:15:41.310521 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716b058f-0efb-4d60-8720-1782c9735e4d None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:15:41.310816 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716b058f-0efb-4d60-8720-1782c9735e4d None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:15:41.311098 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-716b058f-0efb-4d60-8720-1782c9735e4d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:15:41.329674 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-716b058f-0efb-4d60-8720-1782c9735e4d None None] MFA Rules not processed for user `b866bacdafae4db6a32714c07c56515f`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 22:15:41.330906 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-716b058f-0efb-4d60-8720-1782c9735e4d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:15:41.338765 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1127/2247] 10.222.0.52 () {58 vars in 967 bytes} [Wed Jul 13 22:15:41 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 22:15:41.352635 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bddce29-ec7c-4c68-b5ba-a0fc14d0466a service swift] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:15:41.364228 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7bddce29-ec7c-4c68-b5ba-a0fc14d0466a service swift] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:15:41.369679 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3516160-510b-46eb-8fe9-3466ca0050b9 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eb1525233ba48c99b15d4cb75db4822', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417e3e2f1333481795a089bef2e24bea', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:15:41.370924 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3516160-510b-46eb-8fe9-3466ca0050b9 service swift] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:15:41.371411 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3516160-510b-46eb-8fe9-3466ca0050b9 service swift] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:15:41.371799 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c3516160-510b-46eb-8fe9-3466ca0050b9 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:15:41.397122 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3516160-510b-46eb-8fe9-3466ca0050b9 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:15:41.400337 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c3516160-510b-46eb-8fe9-3466ca0050b9 service swift] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:15:41.415075 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1121/2248] 10.222.0.52 () {60 vars in 1384 bytes} [Wed Jul 13 22:15:41 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 63 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 13 22:15:41.696039 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a93615-17af-4271-aacc-545c747dc328 None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:15:41.696551 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a93615-17af-4271-aacc-545c747dc328 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:15:41.696650 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-92a93615-17af-4271-aacc-545c747dc328 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:15:41.714804 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-92a93615-17af-4271-aacc-545c747dc328 None None] MFA Rules not processed for user `b866bacdafae4db6a32714c07c56515f`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 22:15:41.716153 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-92a93615-17af-4271-aacc-545c747dc328 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:15:41.723897 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1128/2249] 10.222.0.52 () {58 vars in 1009 bytes} [Wed Jul 13 22:15:41 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 22:15:41.741330 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3516160-510b-46eb-8fe9-3466ca0050b9 service swift] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:15:41.753283 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c3516160-510b-46eb-8fe9-3466ca0050b9 service swift] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:15:41.758384 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae7a210-95b5-40d7-9ec8-9578bc042de3 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eb1525233ba48c99b15d4cb75db4822', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417e3e2f1333481795a089bef2e24bea', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:15:41.759083 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae7a210-95b5-40d7-9ec8-9578bc042de3 service swift] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:15:41.759366 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae7a210-95b5-40d7-9ec8-9578bc042de3 service swift] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:15:41.759644 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fae7a210-95b5-40d7-9ec8-9578bc042de3 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:15:41.769621 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae7a210-95b5-40d7-9ec8-9578bc042de3 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:15:41.771272 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fae7a210-95b5-40d7-9ec8-9578bc042de3 service swift] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:15:41.779742 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1122/2250] 10.222.0.52 () {60 vars in 1384 bytes} [Wed Jul 13 22:15:41 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 39 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 13 22:15:42.115803 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ed270b-7055-4fa6-b26b-35966fa26cee None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:15:42.116091 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ed270b-7055-4fa6-b26b-35966fa26cee None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:15:42.116369 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-34ed270b-7055-4fa6-b26b-35966fa26cee None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:15:42.134958 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-34ed270b-7055-4fa6-b26b-35966fa26cee None None] MFA Rules not processed for user `b866bacdafae4db6a32714c07c56515f`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 22:15:42.136175 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-34ed270b-7055-4fa6-b26b-35966fa26cee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:15:42.143994 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1129/2251] 10.222.0.52 () {58 vars in 967 bytes} [Wed Jul 13 22:15:42 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 13 22:15:42.156811 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae7a210-95b5-40d7-9ec8-9578bc042de3 service swift] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:15:42.167664 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fae7a210-95b5-40d7-9ec8-9578bc042de3 service swift] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:15:42.173339 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4742023d-7371-4b6e-bbbe-3cddbdfa339a service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eb1525233ba48c99b15d4cb75db4822', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417e3e2f1333481795a089bef2e24bea', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:15:42.173982 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4742023d-7371-4b6e-bbbe-3cddbdfa339a service swift] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:15:42.174264 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4742023d-7371-4b6e-bbbe-3cddbdfa339a service swift] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:15:42.174632 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4742023d-7371-4b6e-bbbe-3cddbdfa339a service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:15:42.183115 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4742023d-7371-4b6e-bbbe-3cddbdfa339a service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:15:42.184743 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4742023d-7371-4b6e-bbbe-3cddbdfa339a service swift] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:15:42.193527 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1123/2252] 10.222.0.52 () {60 vars in 1384 bytes} [Wed Jul 13 22:15:42 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 13 22:15:49.651213 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ed270b-7055-4fa6-b26b-35966fa26cee None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:15:49.652907 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-34ed270b-7055-4fa6-b26b-35966fa26cee None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:15:49.721079 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-34ed270b-7055-4fa6-b26b-35966fa26cee None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:15:49.725744 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f7c222-4049-4eb0-8abc-23a9d6d5b78b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:15:49.726683 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f7c222-4049-4eb0-8abc-23a9d6d5b78b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:15:49.726958 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f7c222-4049-4eb0-8abc-23a9d6d5b78b admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:15:49.727230 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-74f7c222-4049-4eb0-8abc-23a9d6d5b78b admin admin] PATH_INFO: `/v3/users/619af5dee82a4342a1bca6aee37b9b69` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:15:49.728497 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f7c222-4049-4eb0-8abc-23a9d6d5b78b admin admin] RBAC: Authorizing `identity:delete_user(user_id=619af5dee82a4342a1bca6aee37b9b69)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:15:49.730397 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-74f7c222-4049-4eb0-8abc-23a9d6d5b78b admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:15:49.786907 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-74f7c222-4049-4eb0-8abc-23a9d6d5b78b admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '619af5dee82a4342a1bca6aee37b9b69'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:15:49.795846 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-74f7c222-4049-4eb0-8abc-23a9d6d5b78b admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '619af5dee82a4342a1bca6aee37b9b69'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:15:49.814291 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-74f7c222-4049-4eb0-8abc-23a9d6d5b78b admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '619af5dee82a4342a1bca6aee37b9b69'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:15:49.815697 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-74f7c222-4049-4eb0-8abc-23a9d6d5b78b admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '619af5dee82a4342a1bca6aee37b9b69'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:15:49.816523 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-74f7c222-4049-4eb0-8abc-23a9d6d5b78b admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '619af5dee82a4342a1bca6aee37b9b69'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:15:49.829557 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1130/2253] 10.222.0.52 () {58 vars in 1223 bytes} [Wed Jul 13 22:15:49 2022] DELETE /identity/v3/users/619af5dee82a4342a1bca6aee37b9b69 => generated 0 bytes in 179 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 22:15:49.834763 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404dfde0-068d-447c-9ee0-3e53cd67ffdf None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:15:49.835466 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404dfde0-068d-447c-9ee0-3e53cd67ffdf None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:15:49.835466 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-404dfde0-068d-447c-9ee0-3e53cd67ffdf None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:15:49.853336 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-404dfde0-068d-447c-9ee0-3e53cd67ffdf None None] MFA Rules not processed for user `cc99c93da5074cf5a48bc2aa9a92c5e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 22:15:49.856032 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-404dfde0-068d-447c-9ee0-3e53cd67ffdf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:15:49.927952 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1124/2254] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jul 13 22:15:49 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 22:15:49.939852 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f7c222-4049-4eb0-8abc-23a9d6d5b78b admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:15:49.941323 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-74f7c222-4049-4eb0-8abc-23a9d6d5b78b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:15:49.949433 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:15:50.004677 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-74f7c222-4049-4eb0-8abc-23a9d6d5b78b admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:15:50.008520 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9b1d950-c491-4da3-9e29-3b5901015814 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '847ec9a1bd4d4dec804318a5c845895a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417e3e2f1333481795a089bef2e24bea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:15:50.009187 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b1d950-c491-4da3-9e29-3b5901015814 service neutron] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:15:50.009469 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b1d950-c491-4da3-9e29-3b5901015814 service neutron] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:15:50.009746 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9b1d950-c491-4da3-9e29-3b5901015814 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:15:50.019445 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b1d950-c491-4da3-9e29-3b5901015814 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:15:50.020856 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9b1d950-c491-4da3-9e29-3b5901015814 service neutron] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:15:50.032595 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1131/2255] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jul 13 22:15:49 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 22:15:50.202087 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404dfde0-068d-447c-9ee0-3e53cd67ffdf None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:15:50.203988 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-404dfde0-068d-447c-9ee0-3e53cd67ffdf None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:15:50.218783 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-404dfde0-068d-447c-9ee0-3e53cd67ffdf None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:15:50.225632 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f97147-8f83-4921-b640-bfc8a2637194 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:15:50.226588 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f97147-8f83-4921-b640-bfc8a2637194 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:15:50.226897 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f97147-8f83-4921-b640-bfc8a2637194 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:15:50.227130 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-60f97147-8f83-4921-b640-bfc8a2637194 admin admin] PATH_INFO: `/v3/projects/b59379c9d3b34c51b4af356ae41eb3fe` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:15:50.228436 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f97147-8f83-4921-b640-bfc8a2637194 admin admin] RBAC: Authorizing `identity:delete_project(project_id=b59379c9d3b34c51b4af356ae41eb3fe)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:15:50.230419 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-60f97147-8f83-4921-b640-bfc8a2637194 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:15:50.278519 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:15:50.281716 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-60f97147-8f83-4921-b640-bfc8a2637194 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'b59379c9d3b34c51b4af356ae41eb3fe', 'request_id': 'req-60f97147-8f83-4921-b640-bfc8a2637194'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:15:50.288653 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-60f97147-8f83-4921-b640-bfc8a2637194 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'b59379c9d3b34c51b4af356ae41eb3fe', 'request_id': 'req-60f97147-8f83-4921-b640-bfc8a2637194'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:15:50.294335 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-60f97147-8f83-4921-b640-bfc8a2637194 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'b59379c9d3b34c51b4af356ae41eb3fe', 'request_id': 'req-60f97147-8f83-4921-b640-bfc8a2637194'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:15:50.308085 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-60f97147-8f83-4921-b640-bfc8a2637194 admin admin] The token cache is being invalidate because project b59379c9d3b34c51b4af356ae41eb3fe was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80382) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 22:15:50.308372 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-60f97147-8f83-4921-b640-bfc8a2637194 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:15:50.309012 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-60f97147-8f83-4921-b640-bfc8a2637194 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:15:50.310593 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1125/2256] 10.222.0.52 () {58 vars in 1232 bytes} [Wed Jul 13 22:15:50 2022] DELETE /identity/v3/projects/b59379c9d3b34c51b4af356ae41eb3fe => generated 0 bytes in 109 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 22:15:50.964126 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53b3b8f-51ad-46f1-bc70-99e18abb44c0 None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:15:50.964981 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53b3b8f-51ad-46f1-bc70-99e18abb44c0 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:15:50.965276 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e53b3b8f-51ad-46f1-bc70-99e18abb44c0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:15:50.989216 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-e53b3b8f-51ad-46f1-bc70-99e18abb44c0 None None] MFA Rules not processed for user `b866bacdafae4db6a32714c07c56515f`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 22:15:50.990579 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-e53b3b8f-51ad-46f1-bc70-99e18abb44c0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:15:51.054263 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1132/2257] 10.222.0.52 () {58 vars in 1009 bytes} [Wed Jul 13 22:15:50 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 22:15:51.068732 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f97147-8f83-4921-b640-bfc8a2637194 admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:15:51.070158 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-60f97147-8f83-4921-b640-bfc8a2637194 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:15:51.171638 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-60f97147-8f83-4921-b640-bfc8a2637194 admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:15:51.176798 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4238cee7-337b-4d0a-8019-22ac1c1703b7 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eb1525233ba48c99b15d4cb75db4822', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417e3e2f1333481795a089bef2e24bea', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:15:51.177460 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4238cee7-337b-4d0a-8019-22ac1c1703b7 service swift] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:15:51.178325 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4238cee7-337b-4d0a-8019-22ac1c1703b7 service swift] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:15:51.178652 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4238cee7-337b-4d0a-8019-22ac1c1703b7 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:15:51.188573 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4238cee7-337b-4d0a-8019-22ac1c1703b7 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:15:51.190204 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4238cee7-337b-4d0a-8019-22ac1c1703b7 service swift] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:15:51.199554 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1126/2258] 10.222.0.52 () {60 vars in 1384 bytes} [Wed Jul 13 22:15:51 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 131 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 13 22:15:56.305565 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d524db6-e283-4041-9186-43279abef182 None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:15:56.306180 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d524db6-e283-4041-9186-43279abef182 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:15:56.306551 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d524db6-e283-4041-9186-43279abef182 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:15:56.327996 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-1d524db6-e283-4041-9186-43279abef182 None None] MFA Rules not processed for user `e46d01dd355c4f0b9d175d139c285e3f`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 22:15:56.329458 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-1d524db6-e283-4041-9186-43279abef182 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:15:56.398330 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1133/2259] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jul 13 22:15:56 2022] POST /identity/v3/auth/tokens => generated 3248 bytes in 90 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 22:16:01.195465 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:DELETE=4,keystone:SELECT=39,keystone:INSERT=1 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:16:06.386720 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=74 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:16:09.257772 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14962334-2d0b-449f-b4dd-95c06b10301a None None] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:16:09.258379 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14962334-2d0b-449f-b4dd-95c06b10301a None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:16:09.258379 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-14962334-2d0b-449f-b4dd-95c06b10301a None None] PATH_INFO: `/` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:16:09.259701 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1127/2260] 10.222.0.52 () {54 vars in 921 bytes} [Wed Jul 13 22:16:09 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 22:16:09.266059 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65cf037-9a94-49ac-8ba7-a78ce4256774 None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:16:09.266571 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65cf037-9a94-49ac-8ba7-a78ce4256774 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:16:09.266742 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b65cf037-9a94-49ac-8ba7-a78ce4256774 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:16:09.286474 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-b65cf037-9a94-49ac-8ba7-a78ce4256774 None None] MFA Rules not processed for user `f84281091687464990ffbca590b9343b`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 22:16:09.287726 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-b65cf037-9a94-49ac-8ba7-a78ce4256774 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:16:09.356170 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1134/2261] 10.222.0.52 () {58 vars in 1018 bytes} [Wed Jul 13 22:16:09 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 91 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 13 22:16:09.368368 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14962334-2d0b-449f-b4dd-95c06b10301a None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:16:09.370530 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-14962334-2d0b-449f-b4dd-95c06b10301a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:16:09.441422 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-14962334-2d0b-449f-b4dd-95c06b10301a None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:16:09.447645 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a688ff50-9c30-41a2-8957-ca125312110e service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4d35d50e15d43d3afc59206b8165f30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417e3e2f1333481795a089bef2e24bea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:16:09.448517 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a688ff50-9c30-41a2-8957-ca125312110e service glance] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:16:09.448804 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a688ff50-9c30-41a2-8957-ca125312110e service glance] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:16:09.449098 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a688ff50-9c30-41a2-8957-ca125312110e service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:16:09.460607 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a688ff50-9c30-41a2-8957-ca125312110e service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:16:09.462057 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a688ff50-9c30-41a2-8957-ca125312110e service glance] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:16:09.472580 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1128/2262] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jul 13 22:16:09 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 105 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 22:16:09.553514 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f765210-e147-4359-80cd-2f61d41c15ed None None] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:16:09.554093 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f765210-e147-4359-80cd-2f61d41c15ed None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:16:09.554830 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f765210-e147-4359-80cd-2f61d41c15ed None None] PATH_INFO: `/` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:16:09.555880 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1135/2263] 10.222.0.52 () {54 vars in 921 bytes} [Wed Jul 13 22:16:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 22:16:09.561529 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b91a14-4a87-467b-b26a-d2ebd3c1a037 None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:16:09.561967 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b91a14-4a87-467b-b26a-d2ebd3c1a037 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:16:09.562261 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-69b91a14-4a87-467b-b26a-d2ebd3c1a037 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:16:09.582952 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-69b91a14-4a87-467b-b26a-d2ebd3c1a037 None None] MFA Rules not processed for user `847ec9a1bd4d4dec804318a5c845895a`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 22:16:09.584182 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-69b91a14-4a87-467b-b26a-d2ebd3c1a037 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:16:09.638433 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1129/2264] 10.222.0.52 () {58 vars in 1018 bytes} [Wed Jul 13 22:16:09 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 22:16:09.650643 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f765210-e147-4359-80cd-2f61d41c15ed None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:16:09.652489 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-4f765210-e147-4359-80cd-2f61d41c15ed None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:16:09.666174 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f765210-e147-4359-80cd-2f61d41c15ed None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:16:09.669847 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fe772caf-9b21-4bbb-b78b-210e9534f0dd service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '847ec9a1bd4d4dec804318a5c845895a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417e3e2f1333481795a089bef2e24bea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:16:09.670519 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe772caf-9b21-4bbb-b78b-210e9534f0dd service neutron] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:16:09.670827 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe772caf-9b21-4bbb-b78b-210e9534f0dd service neutron] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:16:09.671105 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fe772caf-9b21-4bbb-b78b-210e9534f0dd service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:16:09.678892 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe772caf-9b21-4bbb-b78b-210e9534f0dd service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:16:09.680490 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fe772caf-9b21-4bbb-b78b-210e9534f0dd service neutron] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:16:09.688537 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1136/2265] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jul 13 22:16:09 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 22:16:09.784558 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e724129f-7fce-4c19-9621-39fb36113723 None None] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:16:09.784843 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e724129f-7fce-4c19-9621-39fb36113723 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:16:09.785115 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e724129f-7fce-4c19-9621-39fb36113723 None None] PATH_INFO: `/` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:16:09.786605 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1130/2266] 10.222.0.52 () {54 vars in 921 bytes} [Wed Jul 13 22:16:09 2022] GET /identity => generated 270 bytes in 4 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 22:16:09.792052 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92d9ecb-73d5-4917-8fc6-c010753ebef1 None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:16:09.792481 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92d9ecb-73d5-4917-8fc6-c010753ebef1 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:16:09.792812 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a92d9ecb-73d5-4917-8fc6-c010753ebef1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:16:09.812172 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-a92d9ecb-73d5-4917-8fc6-c010753ebef1 None None] MFA Rules not processed for user `f84281091687464990ffbca590b9343b`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 22:16:09.813360 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-a92d9ecb-73d5-4917-8fc6-c010753ebef1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:16:09.824737 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1137/2267] 10.222.0.52 () {58 vars in 1018 bytes} [Wed Jul 13 22:16:09 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 22:16:09.835844 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e724129f-7fce-4c19-9621-39fb36113723 None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:16:09.837621 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-e724129f-7fce-4c19-9621-39fb36113723 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:16:09.852221 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e724129f-7fce-4c19-9621-39fb36113723 None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:16:09.856045 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1fdadfc-e875-4b4b-a32c-dc0d0e0d2879 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '847ec9a1bd4d4dec804318a5c845895a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417e3e2f1333481795a089bef2e24bea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:16:09.856688 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fdadfc-e875-4b4b-a32c-dc0d0e0d2879 service neutron] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:16:09.856970 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fdadfc-e875-4b4b-a32c-dc0d0e0d2879 service neutron] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:16:09.857244 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1fdadfc-e875-4b4b-a32c-dc0d0e0d2879 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:16:09.867946 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1fdadfc-e875-4b4b-a32c-dc0d0e0d2879 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:16:09.869426 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d1fdadfc-e875-4b4b-a32c-dc0d0e0d2879 service neutron] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:16:09.880394 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1131/2268] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jul 13 22:16:09 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 22:16:13.857878 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3bef8f-4174-4847-95fe-939aa0c3a79a None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:16:13.858475 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3bef8f-4174-4847-95fe-939aa0c3a79a None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:16:13.858475 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-6a3bef8f-4174-4847-95fe-939aa0c3a79a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:16:13.880254 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-6a3bef8f-4174-4847-95fe-939aa0c3a79a None None] MFA Rules not processed for user `064dcac8f59c431cb7d6848f4aaac347`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 22:16:13.881457 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-6a3bef8f-4174-4847-95fe-939aa0c3a79a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:16:13.941579 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1138/2269] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jul 13 22:16:13 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 22:16:14.585308 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127605fc-a403-4b49-9c44-bd5fe1b6c55b None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:16:14.585818 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127605fc-a403-4b49-9c44-bd5fe1b6c55b None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:16:14.585912 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-127605fc-a403-4b49-9c44-bd5fe1b6c55b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:16:14.605837 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-127605fc-a403-4b49-9c44-bd5fe1b6c55b None None] MFA Rules not processed for user `b866bacdafae4db6a32714c07c56515f`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 22:16:14.607065 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-127605fc-a403-4b49-9c44-bd5fe1b6c55b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:16:14.614728 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1132/2270] 10.222.0.52 () {58 vars in 1009 bytes} [Wed Jul 13 22:16:14 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 22:16:14.627253 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3bef8f-4174-4847-95fe-939aa0c3a79a None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:16:14.639210 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-6a3bef8f-4174-4847-95fe-939aa0c3a79a None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:16:14.644274 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8bbe82-8202-4fac-a0f5-c9dfe57e679d service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eb1525233ba48c99b15d4cb75db4822', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417e3e2f1333481795a089bef2e24bea', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:16:14.645054 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8bbe82-8202-4fac-a0f5-c9dfe57e679d service swift] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:16:14.645340 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8bbe82-8202-4fac-a0f5-c9dfe57e679d service swift] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:16:14.645618 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f8bbe82-8202-4fac-a0f5-c9dfe57e679d service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:16:14.653774 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8bbe82-8202-4fac-a0f5-c9dfe57e679d service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:16:14.655554 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f8bbe82-8202-4fac-a0f5-c9dfe57e679d service swift] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:16:14.663692 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1139/2271] 10.222.0.52 () {60 vars in 1384 bytes} [Wed Jul 13 22:16:14 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 37 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 13 22:16:24.598627 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=58 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:16:24.660581 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=71 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:16:56.314828 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7999043b-1630-410b-bcb2-56c54a7aab7a None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:16:56.315445 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7999043b-1630-410b-bcb2-56c54a7aab7a None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:16:56.315445 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7999043b-1630-410b-bcb2-56c54a7aab7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:16:56.335138 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-7999043b-1630-410b-bcb2-56c54a7aab7a None None] MFA Rules not processed for user `b866bacdafae4db6a32714c07c56515f`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 22:16:56.337047 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-7999043b-1630-410b-bcb2-56c54a7aab7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:16:56.344561 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1133/2272] 10.222.0.52 () {58 vars in 967 bytes} [Wed Jul 13 22:16:56 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 22:16:56.368886 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8bbe82-8202-4fac-a0f5-c9dfe57e679d service swift] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:16:56.379702 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f8bbe82-8202-4fac-a0f5-c9dfe57e679d service swift] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:16:56.384878 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466b1262-25af-4c53-8914-028ee9b4591b service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eb1525233ba48c99b15d4cb75db4822', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417e3e2f1333481795a089bef2e24bea', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:16:56.385758 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466b1262-25af-4c53-8914-028ee9b4591b service swift] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:16:56.386030 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466b1262-25af-4c53-8914-028ee9b4591b service swift] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:16:56.386324 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-466b1262-25af-4c53-8914-028ee9b4591b service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:16:56.396664 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-466b1262-25af-4c53-8914-028ee9b4591b service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:16:56.398295 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-466b1262-25af-4c53-8914-028ee9b4591b service swift] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:16:56.406432 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1140/2273] 10.222.0.52 () {60 vars in 1384 bytes} [Wed Jul 13 22:16:56 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 38 msecs (HTTP/1.1 200) 6 headers in 379 bytes (2 switches on core 0) Jul 13 22:17:06.049206 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7999043b-1630-410b-bcb2-56c54a7aab7a None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:17:06.060549 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7999043b-1630-410b-bcb2-56c54a7aab7a None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:17:06.065407 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8e6b91-80f3-4429-a2e4-26f78b9cad78 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '847ec9a1bd4d4dec804318a5c845895a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417e3e2f1333481795a089bef2e24bea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:17:06.066083 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8e6b91-80f3-4429-a2e4-26f78b9cad78 service neutron] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:17:06.066359 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8e6b91-80f3-4429-a2e4-26f78b9cad78 service neutron] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:17:06.066657 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff8e6b91-80f3-4429-a2e4-26f78b9cad78 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:17:06.068989 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-ff8e6b91-80f3-4429-a2e4-26f78b9cad78 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:17:06.084568 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff8e6b91-80f3-4429-a2e4-26f78b9cad78 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:17:06.085990 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff8e6b91-80f3-4429-a2e4-26f78b9cad78 service neutron] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:17:06.097553 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1134/2274] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jul 13 22:17:06 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 49 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 22:17:06.403493 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=6 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:17:07.294179 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466b1262-25af-4c53-8914-028ee9b4591b service swift] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:17:07.306273 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-466b1262-25af-4c53-8914-028ee9b4591b service swift] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:17:07.310905 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b894dbb3-4fa6-4809-b568-b1afdfcb843e service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '847ec9a1bd4d4dec804318a5c845895a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417e3e2f1333481795a089bef2e24bea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:17:07.311548 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b894dbb3-4fa6-4809-b568-b1afdfcb843e service neutron] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:17:07.311826 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b894dbb3-4fa6-4809-b568-b1afdfcb843e service neutron] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:17:07.312215 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b894dbb3-4fa6-4809-b568-b1afdfcb843e service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:17:07.314322 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-b894dbb3-4fa6-4809-b568-b1afdfcb843e service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:17:07.331971 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b894dbb3-4fa6-4809-b568-b1afdfcb843e service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:17:07.334068 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b894dbb3-4fa6-4809-b568-b1afdfcb843e service neutron] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:17:07.351269 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1141/2275] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jul 13 22:17:07 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 58 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 22:17:16.092216 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=9 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:17:17.343311 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=6 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:17:23.013844 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8e6b91-80f3-4429-a2e4-26f78b9cad78 service neutron] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:17:23.015541 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-ff8e6b91-80f3-4429-a2e4-26f78b9cad78 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:17:23.030323 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff8e6b91-80f3-4429-a2e4-26f78b9cad78 service neutron] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:17:23.037297 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d0d7a74c-057e-4a94-941c-49aea0e1e239 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84281091687464990ffbca590b9343b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417e3e2f1333481795a089bef2e24bea', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:17:23.038110 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d7a74c-057e-4a94-941c-49aea0e1e239 service nova] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:17:23.038766 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d7a74c-057e-4a94-941c-49aea0e1e239 service nova] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:17:23.039043 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d0d7a74c-057e-4a94-941c-49aea0e1e239 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:17:23.040938 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-d0d7a74c-057e-4a94-941c-49aea0e1e239 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:17:23.108649 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d7a74c-057e-4a94-941c-49aea0e1e239 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:17:23.110188 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d0d7a74c-057e-4a94-941c-49aea0e1e239 service nova] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:17:23.119694 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1135/2276] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jul 13 22:17:23 2022] GET /identity/v3/auth/tokens => generated 3242 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 22:17:30.296167 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1246f80-ceea-45ef-961c-29036192ceac None None] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:17:30.296731 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1246f80-ceea-45ef-961c-29036192ceac None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:17:30.296731 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1246f80-ceea-45ef-961c-29036192ceac None None] PATH_INFO: `/` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:17:30.298102 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1142/2277] 10.222.0.52 () {54 vars in 921 bytes} [Wed Jul 13 22:17:30 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 22:17:30.307577 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88791ecb-26c9-409d-b6cc-8887b0dfbcd1 None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:17:30.308108 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88791ecb-26c9-409d-b6cc-8887b0dfbcd1 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:17:30.308108 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-88791ecb-26c9-409d-b6cc-8887b0dfbcd1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:17:30.327320 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-88791ecb-26c9-409d-b6cc-8887b0dfbcd1 None None] MFA Rules not processed for user `ac673d641d8849c79bf4cf71c9805587`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 22:17:30.328571 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-88791ecb-26c9-409d-b6cc-8887b0dfbcd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:17:30.380406 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1246f80-ceea-45ef-961c-29036192ceac None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:17:30.382168 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-d1246f80-ceea-45ef-961c-29036192ceac None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:17:30.405195 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1136/2278] 10.222.0.52 () {58 vars in 1018 bytes} [Wed Jul 13 22:17:30 2022] POST /identity/v3/auth/tokens => generated 3298 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 22:17:30.417443 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88791ecb-26c9-409d-b6cc-8887b0dfbcd1 None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:17:30.418832 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-88791ecb-26c9-409d-b6cc-8887b0dfbcd1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:17:30.432285 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-88791ecb-26c9-409d-b6cc-8887b0dfbcd1 None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:17:30.437490 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a322fece-f671-43d3-a0c4-444c5c9bbc6e service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac673d641d8849c79bf4cf71c9805587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417e3e2f1333481795a089bef2e24bea', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:17:30.438142 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a322fece-f671-43d3-a0c4-444c5c9bbc6e service placement] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:17:30.438421 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a322fece-f671-43d3-a0c4-444c5c9bbc6e service placement] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:17:30.438731 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a322fece-f671-43d3-a0c4-444c5c9bbc6e service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:17:30.448001 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a322fece-f671-43d3-a0c4-444c5c9bbc6e service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:17:30.449608 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a322fece-f671-43d3-a0c4-444c5c9bbc6e service placement] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:17:30.453828 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1246f80-ceea-45ef-961c-29036192ceac None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:17:30.458472 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f453de4-d3d2-4505-ba8f-78aa4d1049f3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:17:30.459239 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1137/2279] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jul 13 22:17:30 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 22:17:30.459446 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f453de4-d3d2-4505-ba8f-78aa4d1049f3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:17:30.459731 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f453de4-d3d2-4505-ba8f-78aa4d1049f3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:17:30.460003 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f453de4-d3d2-4505-ba8f-78aa4d1049f3 admin admin] PATH_INFO: `/v3/users/e46d01dd355c4f0b9d175d139c285e3f` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:17:30.461309 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f453de4-d3d2-4505-ba8f-78aa4d1049f3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=e46d01dd355c4f0b9d175d139c285e3f)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:17:30.463406 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3f453de4-d3d2-4505-ba8f-78aa4d1049f3 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:17:30.513398 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-3f453de4-d3d2-4505-ba8f-78aa4d1049f3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': 'e46d01dd355c4f0b9d175d139c285e3f'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:17:30.521949 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-3f453de4-d3d2-4505-ba8f-78aa4d1049f3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': 'e46d01dd355c4f0b9d175d139c285e3f'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:17:30.533727 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-3f453de4-d3d2-4505-ba8f-78aa4d1049f3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': 'e46d01dd355c4f0b9d175d139c285e3f'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:17:30.534372 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-3f453de4-d3d2-4505-ba8f-78aa4d1049f3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': 'e46d01dd355c4f0b9d175d139c285e3f'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:17:30.535011 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-3f453de4-d3d2-4505-ba8f-78aa4d1049f3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': 'e46d01dd355c4f0b9d175d139c285e3f'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:17:30.547736 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1143/2280] 10.222.0.52 () {58 vars in 1223 bytes} [Wed Jul 13 22:17:30 2022] DELETE /identity/v3/users/e46d01dd355c4f0b9d175d139c285e3f => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 22:17:30.732712 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a322fece-f671-43d3-a0c4-444c5c9bbc6e service placement] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:17:30.743535 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a322fece-f671-43d3-a0c4-444c5c9bbc6e service placement] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:17:30.809044 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd7b200-1185-4ac0-9aee-08dd6beb3ebb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:17:30.809922 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd7b200-1185-4ac0-9aee-08dd6beb3ebb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:17:30.810200 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd7b200-1185-4ac0-9aee-08dd6beb3ebb admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:17:30.810473 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5cd7b200-1185-4ac0-9aee-08dd6beb3ebb admin admin] PATH_INFO: `/v3/projects/6587cfa2404644d5bf2100f2a49db1aa` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:17:30.811783 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd7b200-1185-4ac0-9aee-08dd6beb3ebb admin admin] RBAC: Authorizing `identity:delete_project(project_id=6587cfa2404644d5bf2100f2a49db1aa)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:17:30.813688 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5cd7b200-1185-4ac0-9aee-08dd6beb3ebb admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:17:30.839652 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=99,keystone:DELETE=1 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:17:30.865758 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-5cd7b200-1185-4ac0-9aee-08dd6beb3ebb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '6587cfa2404644d5bf2100f2a49db1aa', 'request_id': 'req-5cd7b200-1185-4ac0-9aee-08dd6beb3ebb'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:17:30.872609 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-5cd7b200-1185-4ac0-9aee-08dd6beb3ebb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '6587cfa2404644d5bf2100f2a49db1aa', 'request_id': 'req-5cd7b200-1185-4ac0-9aee-08dd6beb3ebb'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:17:30.878545 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-5cd7b200-1185-4ac0-9aee-08dd6beb3ebb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '6587cfa2404644d5bf2100f2a49db1aa', 'request_id': 'req-5cd7b200-1185-4ac0-9aee-08dd6beb3ebb'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:17:30.894012 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-5cd7b200-1185-4ac0-9aee-08dd6beb3ebb admin admin] The token cache is being invalidate because project 6587cfa2404644d5bf2100f2a49db1aa was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80382) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 22:17:30.894296 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-5cd7b200-1185-4ac0-9aee-08dd6beb3ebb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:17:30.894904 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-5cd7b200-1185-4ac0-9aee-08dd6beb3ebb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:17:30.896453 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1138/2281] 10.222.0.52 () {58 vars in 1232 bytes} [Wed Jul 13 22:17:30 2022] DELETE /identity/v3/projects/6587cfa2404644d5bf2100f2a49db1aa => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 22:17:40.543960 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=53,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:17:40.890901 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=15,keystone:DELETE=6,keystone:INSERT=1 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:17:56.070322 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f453de4-d3d2-4505-ba8f-78aa4d1049f3 admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:17:56.072932 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-3f453de4-d3d2-4505-ba8f-78aa4d1049f3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:17:56.150212 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f453de4-d3d2-4505-ba8f-78aa4d1049f3 admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:17:56.216480 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b543d2-1e40-496b-9793-274923c42e18 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84281091687464990ffbca590b9343b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417e3e2f1333481795a089bef2e24bea', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:17:56.217567 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b543d2-1e40-496b-9793-274923c42e18 service nova] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:17:56.217879 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b543d2-1e40-496b-9793-274923c42e18 service nova] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:17:56.218181 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c7b543d2-1e40-496b-9793-274923c42e18 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:17:56.221352 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-c7b543d2-1e40-496b-9793-274923c42e18 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:17:56.286761 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b543d2-1e40-496b-9793-274923c42e18 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:17:56.288903 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c7b543d2-1e40-496b-9793-274923c42e18 service nova] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:17:56.301067 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1144/2282] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jul 13 22:17:56 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 231 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 22:18:04.413050 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd7b200-1185-4ac0-9aee-08dd6beb3ebb admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:18:04.414668 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-5cd7b200-1185-4ac0-9aee-08dd6beb3ebb admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:18:04.474438 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5cd7b200-1185-4ac0-9aee-08dd6beb3ebb admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:18:04.478272 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f37d60a-61a6-499d-8f6b-90269a868652 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '847ec9a1bd4d4dec804318a5c845895a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417e3e2f1333481795a089bef2e24bea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:18:04.478962 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f37d60a-61a6-499d-8f6b-90269a868652 service neutron] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:18:04.479394 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f37d60a-61a6-499d-8f6b-90269a868652 service neutron] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:18:04.479488 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0f37d60a-61a6-499d-8f6b-90269a868652 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:18:04.481343 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-0f37d60a-61a6-499d-8f6b-90269a868652 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:18:04.493487 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f37d60a-61a6-499d-8f6b-90269a868652 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:18:04.493487 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0f37d60a-61a6-499d-8f6b-90269a868652 service neutron] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:18:04.500931 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1139/2283] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jul 13 22:18:04 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 89 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 22:18:06.296329 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=84 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:18:07.727361 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b543d2-1e40-496b-9793-274923c42e18 service nova] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:18:07.729221 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-c7b543d2-1e40-496b-9793-274923c42e18 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:18:07.746180 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c7b543d2-1e40-496b-9793-274923c42e18 service nova] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:18:07.754119 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e5d019f4-0ad1-420d-aaea-3431bba1f897 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84281091687464990ffbca590b9343b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417e3e2f1333481795a089bef2e24bea', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:18:07.754801 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d019f4-0ad1-420d-aaea-3431bba1f897 service nova] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:18:07.755067 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d019f4-0ad1-420d-aaea-3431bba1f897 service nova] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:18:07.755342 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e5d019f4-0ad1-420d-aaea-3431bba1f897 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:18:07.757492 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-e5d019f4-0ad1-420d-aaea-3431bba1f897 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:18:07.817493 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d019f4-0ad1-420d-aaea-3431bba1f897 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:18:07.819052 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e5d019f4-0ad1-420d-aaea-3431bba1f897 service nova] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:18:07.829778 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1145/2284] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jul 13 22:18:07 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 103 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 22:18:13.878294 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f37d60a-61a6-499d-8f6b-90269a868652 service neutron] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:18:13.880252 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-0f37d60a-61a6-499d-8f6b-90269a868652 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:18:13.906617 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0f37d60a-61a6-499d-8f6b-90269a868652 service neutron] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:18:13.913861 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ddd468-0b5c-4bf4-8289-5d4b07487712 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84281091687464990ffbca590b9343b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417e3e2f1333481795a089bef2e24bea', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:18:13.914609 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ddd468-0b5c-4bf4-8289-5d4b07487712 service nova] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:18:13.914897 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ddd468-0b5c-4bf4-8289-5d4b07487712 service nova] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:18:13.915177 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61ddd468-0b5c-4bf4-8289-5d4b07487712 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:18:13.917170 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-61ddd468-0b5c-4bf4-8289-5d4b07487712 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:18:13.930653 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ddd468-0b5c-4bf4-8289-5d4b07487712 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:18:13.932263 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61ddd468-0b5c-4bf4-8289-5d4b07487712 service nova] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:18:13.959290 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1140/2285] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jul 13 22:18:13 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 82 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 22:18:17.543165 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc770057-1ae0-48b5-b9a4-8dc0d736a28b None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:18:17.543745 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc770057-1ae0-48b5-b9a4-8dc0d736a28b None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:18:17.543745 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bc770057-1ae0-48b5-b9a4-8dc0d736a28b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:18:17.563864 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-bc770057-1ae0-48b5-b9a4-8dc0d736a28b None None] MFA Rules not processed for user `83d2f82cf096424aa4979f443a2729d6`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 22:18:17.565071 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-bc770057-1ae0-48b5-b9a4-8dc0d736a28b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:18:17.573382 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1146/2286] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jul 13 22:18:17 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 31 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 22:18:17.915275 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ddd468-0b5c-4bf4-8289-5d4b07487712 service nova] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:18:17.925396 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61ddd468-0b5c-4bf4-8289-5d4b07487712 service nova] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:18:17.929569 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-814cbed5-0855-4a53-8fec-9a733c18da66 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '847ec9a1bd4d4dec804318a5c845895a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417e3e2f1333481795a089bef2e24bea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:18:17.930523 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814cbed5-0855-4a53-8fec-9a733c18da66 service neutron] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:18:17.931022 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814cbed5-0855-4a53-8fec-9a733c18da66 service neutron] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:18:17.931448 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-814cbed5-0855-4a53-8fec-9a733c18da66 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:18:17.933772 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-814cbed5-0855-4a53-8fec-9a733c18da66 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:18:17.944718 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-814cbed5-0855-4a53-8fec-9a733c18da66 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:18:17.946732 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-814cbed5-0855-4a53-8fec-9a733c18da66 service neutron] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:18:17.955418 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1141/2287] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jul 13 22:18:17 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 41 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 22:18:18.289520 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc770057-1ae0-48b5-b9a4-8dc0d736a28b None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:18:18.291194 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-bc770057-1ae0-48b5-b9a4-8dc0d736a28b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:18:18.368585 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bc770057-1ae0-48b5-b9a4-8dc0d736a28b None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:18:18.373677 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f71bb1-a15a-41da-bbc2-08f5cbf8f67a service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac673d641d8849c79bf4cf71c9805587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417e3e2f1333481795a089bef2e24bea', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:18:18.374337 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f71bb1-a15a-41da-bbc2-08f5cbf8f67a service placement] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:18:18.374663 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f71bb1-a15a-41da-bbc2-08f5cbf8f67a service placement] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:18:18.374940 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-23f71bb1-a15a-41da-bbc2-08f5cbf8f67a service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:18:18.376748 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-23f71bb1-a15a-41da-bbc2-08f5cbf8f67a service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:18:18.388459 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f71bb1-a15a-41da-bbc2-08f5cbf8f67a service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:18:18.390059 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-23f71bb1-a15a-41da-bbc2-08f5cbf8f67a service placement] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:18:18.400087 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1147/2288] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jul 13 22:18:18 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 22:18:25.311938 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622e0dc1-4a9b-42ee-86f2-11207b6c92e4 None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:18:25.312458 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622e0dc1-4a9b-42ee-86f2-11207b6c92e4 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:18:25.312458 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-622e0dc1-4a9b-42ee-86f2-11207b6c92e4 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:18:25.331780 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-622e0dc1-4a9b-42ee-86f2-11207b6c92e4 None None] MFA Rules not processed for user `45b5d8b0ba324f629f697cf0de5918bb`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 22:18:25.333337 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-622e0dc1-4a9b-42ee-86f2-11207b6c92e4 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:18:25.341054 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1142/2289] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jul 13 22:18:25 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 22:18:28.395979 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=63 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:18:30.655876 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f71bb1-a15a-41da-bbc2-08f5cbf8f67a service placement] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:18:30.668602 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-23f71bb1-a15a-41da-bbc2-08f5cbf8f67a service placement] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:18:30.674694 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21581152-3e7c-4621-be71-9f12c41b9b58 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84281091687464990ffbca590b9343b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417e3e2f1333481795a089bef2e24bea', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:18:30.675348 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21581152-3e7c-4621-be71-9f12c41b9b58 service nova] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:18:30.675702 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21581152-3e7c-4621-be71-9f12c41b9b58 service nova] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:18:30.675870 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-21581152-3e7c-4621-be71-9f12c41b9b58 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:18:30.678013 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-21581152-3e7c-4621-be71-9f12c41b9b58 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:18:30.691471 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21581152-3e7c-4621-be71-9f12c41b9b58 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:18:30.693469 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-21581152-3e7c-4621-be71-9f12c41b9b58 service nova] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:18:30.704837 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1148/2290] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jul 13 22:18:30 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 50 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 22:18:32.905522 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622e0dc1-4a9b-42ee-86f2-11207b6c92e4 None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:18:32.913347 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=42 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:18:32.921609 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-622e0dc1-4a9b-42ee-86f2-11207b6c92e4 None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:18:32.934196 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aea0d705-5223-4d77-aa1d-fef67261e404 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84281091687464990ffbca590b9343b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417e3e2f1333481795a089bef2e24bea', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:18:32.934911 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea0d705-5223-4d77-aa1d-fef67261e404 service nova] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:18:32.935192 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea0d705-5223-4d77-aa1d-fef67261e404 service nova] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:18:32.935467 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aea0d705-5223-4d77-aa1d-fef67261e404 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:18:32.937367 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-aea0d705-5223-4d77-aa1d-fef67261e404 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:18:32.951110 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea0d705-5223-4d77-aa1d-fef67261e404 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:18:32.952721 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aea0d705-5223-4d77-aa1d-fef67261e404 service nova] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:18:32.965179 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1143/2291] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jul 13 22:18:32 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 60 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 22:18:33.655261 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21581152-3e7c-4621-be71-9f12c41b9b58 service nova] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:18:33.666692 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-21581152-3e7c-4621-be71-9f12c41b9b58 service nova] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:18:33.672596 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13611ab3-2842-48a1-8170-a95c4c399df9 service nova] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'f84281091687464990ffbca590b9343b', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417e3e2f1333481795a089bef2e24bea', 'project_domain_id': 'default', 'roles': ['reader', 'ResellerAdmin', 'admin', 'member', 'service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:18:33.673279 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13611ab3-2842-48a1-8170-a95c4c399df9 service nova] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:18:33.673559 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13611ab3-2842-48a1-8170-a95c4c399df9 service nova] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:18:33.673826 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-13611ab3-2842-48a1-8170-a95c4c399df9 service nova] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:18:33.675662 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-13611ab3-2842-48a1-8170-a95c4c399df9 service nova] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:18:33.737410 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13611ab3-2842-48a1-8170-a95c4c399df9 service nova] RBAC: Authorizing `identity:validate_token()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:18:33.739071 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-13611ab3-2842-48a1-8170-a95c4c399df9 service nova] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:18:33.749517 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1149/2292] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jul 13 22:18:33 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 95 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 22:18:42.960878 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=5 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:18:43.745252 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=34 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:18:47.189658 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563b516b-cffd-46af-a8ef-7723d69b260b None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:18:47.190233 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563b516b-cffd-46af-a8ef-7723d69b260b None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:18:47.190233 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-563b516b-cffd-46af-a8ef-7723d69b260b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:18:47.208316 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-563b516b-cffd-46af-a8ef-7723d69b260b None None] MFA Rules not processed for user `064dcac8f59c431cb7d6848f4aaac347`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 22:18:47.209601 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-563b516b-cffd-46af-a8ef-7723d69b260b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:18:47.216995 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1144/2293] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jul 13 22:18:47 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 28 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 22:18:57.203341 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=3 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:19:55.450130 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13611ab3-2842-48a1-8170-a95c4c399df9 service nova] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:19:55.460433 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-13611ab3-2842-48a1-8170-a95c4c399df9 service nova] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:19:55.464558 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddcd081b-fb84-456c-a899-d30dfc2bb14d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '847ec9a1bd4d4dec804318a5c845895a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417e3e2f1333481795a089bef2e24bea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:19:55.465203 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcd081b-fb84-456c-a899-d30dfc2bb14d service neutron] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:19:55.465482 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcd081b-fb84-456c-a899-d30dfc2bb14d service neutron] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:19:55.465746 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ddcd081b-fb84-456c-a899-d30dfc2bb14d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:19:55.467797 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-ddcd081b-fb84-456c-a899-d30dfc2bb14d service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:19:55.543355 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddcd081b-fb84-456c-a899-d30dfc2bb14d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:19:55.545523 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ddcd081b-fb84-456c-a899-d30dfc2bb14d service neutron] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:19:55.556881 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1150/2294] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jul 13 22:19:55 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 107 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 22:19:59.980533 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563b516b-cffd-46af-a8ef-7723d69b260b None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:19:59.982907 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-563b516b-cffd-46af-a8ef-7723d69b260b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:19:59.998231 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-563b516b-cffd-46af-a8ef-7723d69b260b None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:20:00.003602 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-530d67c0-9385-4499-9761-93da0faecf97 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:20:00.004538 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530d67c0-9385-4499-9761-93da0faecf97 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:20:00.004875 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530d67c0-9385-4499-9761-93da0faecf97 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:20:00.005205 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-530d67c0-9385-4499-9761-93da0faecf97 admin admin] PATH_INFO: `/v3/users/45b5d8b0ba324f629f697cf0de5918bb` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:20:00.006554 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530d67c0-9385-4499-9761-93da0faecf97 admin admin] RBAC: Authorizing `identity:delete_user(user_id=45b5d8b0ba324f629f697cf0de5918bb)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:20:00.008573 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-530d67c0-9385-4499-9761-93da0faecf97 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:20:00.063028 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-530d67c0-9385-4499-9761-93da0faecf97 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '45b5d8b0ba324f629f697cf0de5918bb'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:20:00.073809 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-530d67c0-9385-4499-9761-93da0faecf97 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '45b5d8b0ba324f629f697cf0de5918bb'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:20:00.090962 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-530d67c0-9385-4499-9761-93da0faecf97 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '45b5d8b0ba324f629f697cf0de5918bb'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:20:00.091716 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-530d67c0-9385-4499-9761-93da0faecf97 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '45b5d8b0ba324f629f697cf0de5918bb'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:20:00.092321 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-530d67c0-9385-4499-9761-93da0faecf97 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '45b5d8b0ba324f629f697cf0de5918bb'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:20:00.105966 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1145/2295] 10.222.0.52 () {58 vars in 1223 bytes} [Wed Jul 13 22:19:59 2022] DELETE /identity/v3/users/45b5d8b0ba324f629f697cf0de5918bb => generated 0 bytes in 126 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 22:20:00.301797 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddcd081b-fb84-456c-a899-d30dfc2bb14d service neutron] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:20:00.312143 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ddcd081b-fb84-456c-a899-d30dfc2bb14d service neutron] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:20:00.372632 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2af05efb-8ba8-4341-bcd1-4d4df6301ceb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:20:00.373593 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af05efb-8ba8-4341-bcd1-4d4df6301ceb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:20:00.374021 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af05efb-8ba8-4341-bcd1-4d4df6301ceb admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:20:00.374383 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2af05efb-8ba8-4341-bcd1-4d4df6301ceb admin admin] PATH_INFO: `/v3/projects/75f4f5476c7e451a82f1cfc82a0ee689` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:20:00.375791 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af05efb-8ba8-4341-bcd1-4d4df6301ceb admin admin] RBAC: Authorizing `identity:delete_project(project_id=75f4f5476c7e451a82f1cfc82a0ee689)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:20:00.377733 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2af05efb-8ba8-4341-bcd1-4d4df6301ceb admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:20:00.425482 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-2af05efb-8ba8-4341-bcd1-4d4df6301ceb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '75f4f5476c7e451a82f1cfc82a0ee689', 'request_id': 'req-2af05efb-8ba8-4341-bcd1-4d4df6301ceb'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:20:00.431970 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-2af05efb-8ba8-4341-bcd1-4d4df6301ceb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '75f4f5476c7e451a82f1cfc82a0ee689', 'request_id': 'req-2af05efb-8ba8-4341-bcd1-4d4df6301ceb'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:20:00.437526 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-2af05efb-8ba8-4341-bcd1-4d4df6301ceb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '75f4f5476c7e451a82f1cfc82a0ee689', 'request_id': 'req-2af05efb-8ba8-4341-bcd1-4d4df6301ceb'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:20:00.452282 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-2af05efb-8ba8-4341-bcd1-4d4df6301ceb admin admin] The token cache is being invalidate because project 75f4f5476c7e451a82f1cfc82a0ee689 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80381) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 22:20:00.452576 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-2af05efb-8ba8-4341-bcd1-4d4df6301ceb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:20:00.453281 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-2af05efb-8ba8-4341-bcd1-4d4df6301ceb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:20:00.455175 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1151/2296] 10.222.0.52 () {58 vars in 1232 bytes} [Wed Jul 13 22:20:00 2022] DELETE /identity/v3/projects/75f4f5476c7e451a82f1cfc82a0ee689 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 22:20:00.460834 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0622f8c1-1f07-4055-ae86-9d87701d05df None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:20:00.461102 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0622f8c1-1f07-4055-ae86-9d87701d05df None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:20:00.461400 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0622f8c1-1f07-4055-ae86-9d87701d05df None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:20:00.481403 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-0622f8c1-1f07-4055-ae86-9d87701d05df None None] MFA Rules not processed for user `cc99c93da5074cf5a48bc2aa9a92c5e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 22:20:00.482666 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-0622f8c1-1f07-4055-ae86-9d87701d05df None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:20:00.544827 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1146/2297] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jul 13 22:20:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 85 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 22:20:00.553237 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82943b23-3b67-413f-9986-d80ba564da32 None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:20:00.554146 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82943b23-3b67-413f-9986-d80ba564da32 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:20:00.554146 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-82943b23-3b67-413f-9986-d80ba564da32 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:20:00.572560 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-82943b23-3b67-413f-9986-d80ba564da32 None None] MFA Rules not processed for user `cc99c93da5074cf5a48bc2aa9a92c5e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 22:20:00.573862 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-82943b23-3b67-413f-9986-d80ba564da32 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:20:00.582366 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1152/2298] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jul 13 22:20:00 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 30 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 22:20:00.586802 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0622f8c1-1f07-4055-ae86-9d87701d05df None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:20:00.596873 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0622f8c1-1f07-4055-ae86-9d87701d05df None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:20:00.601695 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d1f43f-02f0-4a07-b347-69ce177a7cd3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:20:00.602587 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d1f43f-02f0-4a07-b347-69ce177a7cd3 admin admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:20:00.602886 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d1f43f-02f0-4a07-b347-69ce177a7cd3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:20:00.603190 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53d1f43f-02f0-4a07-b347-69ce177a7cd3 admin admin] PATH_INFO: `/v3/domains` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:20:00.604024 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d1f43f-02f0-4a07-b347-69ce177a7cd3 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 22:20:00.604413 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d1f43f-02f0-4a07-b347-69ce177a7cd3 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:20:00.606078 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53d1f43f-02f0-4a07-b347-69ce177a7cd3 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:20:00.622907 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1147/2299] 10.222.0.52 () {58 vars in 1152 bytes} [Wed Jul 13 22:20:00 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 22:20:00.624456 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82943b23-3b67-413f-9986-d80ba564da32 None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:20:00.634377 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-82943b23-3b67-413f-9986-d80ba564da32 None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:20:00.639706 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cdfbb6c-40f4-4518-8669-4ede851908f7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:20:00.640382 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdfbb6c-40f4-4518-8669-4ede851908f7 admin admin] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:20:00.641385 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdfbb6c-40f4-4518-8669-4ede851908f7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:20:00.641673 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2cdfbb6c-40f4-4518-8669-4ede851908f7 admin admin] PATH_INFO: `/v3/projects` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:20:00.642385 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cdfbb6c-40f4-4518-8669-4ede851908f7 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:20:00.644357 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2cdfbb6c-40f4-4518-8669-4ede851908f7 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:20:00.659711 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:20:00.678223 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1153/2300] 10.222.0.52 () {60 vars in 1152 bytes} [Wed Jul 13 22:20:00 2022] POST /identity/v3/projects => generated 387 bytes in 54 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 22:20:00.680701 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d1f43f-02f0-4a07-b347-69ce177a7cd3 admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:20:00.697281 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53d1f43f-02f0-4a07-b347-69ce177a7cd3 admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:20:00.760966 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625a8597-9453-4c89-8ef3-572baaf0e3af admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:20:00.761664 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625a8597-9453-4c89-8ef3-572baaf0e3af admin admin] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:20:00.761918 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625a8597-9453-4c89-8ef3-572baaf0e3af admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:20:00.762194 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625a8597-9453-4c89-8ef3-572baaf0e3af admin admin] PATH_INFO: `/v3/users` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:20:00.762919 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-625a8597-9453-4c89-8ef3-572baaf0e3af admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:20:00.764865 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-625a8597-9453-4c89-8ef3-572baaf0e3af admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:20:00.772194 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:20:00.789345 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1148/2301] 10.222.0.52 () {60 vars in 1143 bytes} [Wed Jul 13 22:20:00 2022] POST /identity/v3/users => generated 294 bytes in 109 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 22:20:00.793171 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cdfbb6c-40f4-4518-8669-4ede851908f7 admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:20:00.803945 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2cdfbb6c-40f4-4518-8669-4ede851908f7 admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:20:00.809285 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dfe8036-f554-450d-896c-90f9c6af17cb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:20:00.809961 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfe8036-f554-450d-896c-90f9c6af17cb admin admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:20:00.810260 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfe8036-f554-450d-896c-90f9c6af17cb admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:20:00.810564 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3dfe8036-f554-450d-896c-90f9c6af17cb admin admin] PATH_INFO: `/v3/roles` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:20:00.811343 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dfe8036-f554-450d-896c-90f9c6af17cb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:20:00.813041 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-3dfe8036-f554-450d-896c-90f9c6af17cb admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:20:00.824698 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1154/2302] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jul 13 22:20:00 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 22:20:00.829017 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625a8597-9453-4c89-8ef3-572baaf0e3af admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:20:00.839907 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-625a8597-9453-4c89-8ef3-572baaf0e3af admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:20:00.845553 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4cf8443-12e8-47f7-a79f-b472ccbc9a8d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:20:00.846222 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cf8443-12e8-47f7-a79f-b472ccbc9a8d admin admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:20:00.846531 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cf8443-12e8-47f7-a79f-b472ccbc9a8d admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:20:00.846867 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4cf8443-12e8-47f7-a79f-b472ccbc9a8d admin admin] PATH_INFO: `/v3/roles` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:20:00.847614 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4cf8443-12e8-47f7-a79f-b472ccbc9a8d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:20:00.849226 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a4cf8443-12e8-47f7-a79f-b472ccbc9a8d admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:20:00.860047 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1149/2303] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jul 13 22:20:00 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 13 22:20:00.864566 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dfe8036-f554-450d-896c-90f9c6af17cb admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:20:00.874971 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3dfe8036-f554-450d-896c-90f9c6af17cb admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:20:00.879570 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30650b1b-1581-4898-b42e-efe56243ee30 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:20:00.880460 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30650b1b-1581-4898-b42e-efe56243ee30 admin admin] REQUEST_METHOD: `PUT` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:20:00.880733 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30650b1b-1581-4898-b42e-efe56243ee30 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:20:00.881005 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-30650b1b-1581-4898-b42e-efe56243ee30 admin admin] PATH_INFO: `/v3/projects/c2f2f47e8fae45898130b3410f6cc5ca/users/019717606a664ad4ab4277ac219e875c/roles/2a5e86976394498abb6310697cdb3f8b` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:20:00.896265 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30650b1b-1581-4898-b42e-efe56243ee30 admin admin] RBAC: Authorizing `identity:create_grant(project_id=c2f2f47e8fae45898130b3410f6cc5ca, user_id=019717606a664ad4ab4277ac219e875c, role_id=2a5e86976394498abb6310697cdb3f8b)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:20:00.898941 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-30650b1b-1581-4898-b42e-efe56243ee30 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:20:00.913018 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1155/2304] 10.222.0.52 () {60 vars in 1482 bytes} [Wed Jul 13 22:20:00 2022] PUT /identity/v3/projects/c2f2f47e8fae45898130b3410f6cc5ca/users/019717606a664ad4ab4277ac219e875c/roles/2a5e86976394498abb6310697cdb3f8b => generated 0 bytes in 49 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 22:20:00.923898 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625caba5-c839-4f4b-883d-d8695140cdb0 None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:20:00.924272 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625caba5-c839-4f4b-883d-d8695140cdb0 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:20:00.924565 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-625caba5-c839-4f4b-883d-d8695140cdb0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:20:00.952525 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-625caba5-c839-4f4b-883d-d8695140cdb0 None None] MFA Rules not processed for user `019717606a664ad4ab4277ac219e875c`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 22:20:00.954164 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-625caba5-c839-4f4b-883d-d8695140cdb0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:20:01.021926 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1150/2305] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jul 13 22:20:00 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 100 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 22:20:01.026616 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30650b1b-1581-4898-b42e-efe56243ee30 admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:20:01.084086 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71aaaa2a-346f-4631-90c4-fc6114677d0f None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:20:01.084393 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71aaaa2a-346f-4631-90c4-fc6114677d0f None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:20:01.084691 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71aaaa2a-346f-4631-90c4-fc6114677d0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:20:01.095716 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-30650b1b-1581-4898-b42e-efe56243ee30 admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:20:01.100334 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d901ea65-e384-4684-ab02-47fd8025b5d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:20:01.100997 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d901ea65-e384-4684-ab02-47fd8025b5d2 admin admin] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:20:01.101278 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d901ea65-e384-4684-ab02-47fd8025b5d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:20:01.101552 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d901ea65-e384-4684-ab02-47fd8025b5d2 admin admin] PATH_INFO: `/v3/projects` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:20:01.102038 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-71aaaa2a-346f-4631-90c4-fc6114677d0f None None] MFA Rules not processed for user `18b51e7024fc46a8af3644af27ab886f`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 22:20:01.102253 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d901ea65-e384-4684-ab02-47fd8025b5d2 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:20:01.103258 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-71aaaa2a-346f-4631-90c4-fc6114677d0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:20:01.104250 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d901ea65-e384-4684-ab02-47fd8025b5d2 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:20:01.134318 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1156/2306] 10.222.0.52 () {60 vars in 1152 bytes} [Wed Jul 13 22:20:01 2022] POST /identity/v3/projects => generated 387 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 13 22:20:01.135401 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d901ea65-e384-4684-ab02-47fd8025b5d2 admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:20:01.160716 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1151/2307] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jul 13 22:20:01 2022] POST /identity/v3/auth/tokens => generated 3242 bytes in 78 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 13 22:20:01.198095 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d901ea65-e384-4684-ab02-47fd8025b5d2 admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:20:01.202824 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171c87ae-eefa-48cb-9f2a-853b69540434 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:20:01.203539 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171c87ae-eefa-48cb-9f2a-853b69540434 admin admin] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:20:01.203890 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171c87ae-eefa-48cb-9f2a-853b69540434 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:20:01.204166 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-171c87ae-eefa-48cb-9f2a-853b69540434 admin admin] PATH_INFO: `/v3/users` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:20:01.204865 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171c87ae-eefa-48cb-9f2a-853b69540434 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:20:01.206813 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-171c87ae-eefa-48cb-9f2a-853b69540434 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:20:01.228873 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1157/2308] 10.222.0.52 () {60 vars in 1143 bytes} [Wed Jul 13 22:20:01 2022] POST /identity/v3/users => generated 294 bytes in 94 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 13 22:20:01.232736 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71aaaa2a-346f-4631-90c4-fc6114677d0f None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:20:01.243851 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71aaaa2a-346f-4631-90c4-fc6114677d0f None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:20:01.248674 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71066b10-925a-4c11-bd3b-16c6e8f9f03d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:20:01.249338 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71066b10-925a-4c11-bd3b-16c6e8f9f03d admin admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:20:01.249614 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71066b10-925a-4c11-bd3b-16c6e8f9f03d admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:20:01.249886 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-71066b10-925a-4c11-bd3b-16c6e8f9f03d admin admin] PATH_INFO: `/v3/roles` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:20:01.250701 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71066b10-925a-4c11-bd3b-16c6e8f9f03d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:20:01.252399 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-71066b10-925a-4c11-bd3b-16c6e8f9f03d admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:20:01.262839 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1152/2309] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jul 13 22:20:01 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 22:20:01.266955 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171c87ae-eefa-48cb-9f2a-853b69540434 admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:20:01.277300 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-171c87ae-eefa-48cb-9f2a-853b69540434 admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:20:01.281864 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1b4c1d38-0407-430f-afd2-f5d4321fdb0d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:20:01.282562 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4c1d38-0407-430f-afd2-f5d4321fdb0d admin admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:20:01.282844 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4c1d38-0407-430f-afd2-f5d4321fdb0d admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:20:01.283116 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1b4c1d38-0407-430f-afd2-f5d4321fdb0d admin admin] PATH_INFO: `/v3/roles` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:20:01.283841 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4c1d38-0407-430f-afd2-f5d4321fdb0d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:20:01.285474 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1b4c1d38-0407-430f-afd2-f5d4321fdb0d admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:20:01.295985 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1158/2310] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jul 13 22:20:01 2022] GET /identity/v3/roles => generated 1420 bytes in 30 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 13 22:20:01.300558 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71066b10-925a-4c11-bd3b-16c6e8f9f03d admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:20:01.311496 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-71066b10-925a-4c11-bd3b-16c6e8f9f03d admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:20:01.317498 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205c3022-2a94-4bb9-9c28-f3c376e1b0e3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:20:01.318401 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205c3022-2a94-4bb9-9c28-f3c376e1b0e3 admin admin] REQUEST_METHOD: `PUT` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:20:01.318724 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205c3022-2a94-4bb9-9c28-f3c376e1b0e3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:20:01.319001 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-205c3022-2a94-4bb9-9c28-f3c376e1b0e3 admin admin] PATH_INFO: `/v3/projects/a38b9915ebfb4968939d8d9e87d78638/users/17da521881d4420dac44868b53c55c0f/roles/2a5e86976394498abb6310697cdb3f8b` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:20:01.335989 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205c3022-2a94-4bb9-9c28-f3c376e1b0e3 admin admin] RBAC: Authorizing `identity:create_grant(project_id=a38b9915ebfb4968939d8d9e87d78638, user_id=17da521881d4420dac44868b53c55c0f, role_id=2a5e86976394498abb6310697cdb3f8b)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:20:01.338709 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-205c3022-2a94-4bb9-9c28-f3c376e1b0e3 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:20:01.354275 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1153/2311] 10.222.0.52 () {60 vars in 1482 bytes} [Wed Jul 13 22:20:01 2022] PUT /identity/v3/projects/a38b9915ebfb4968939d8d9e87d78638/users/17da521881d4420dac44868b53c55c0f/roles/2a5e86976394498abb6310697cdb3f8b => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 22:20:01.364924 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b4e7e8-ee65-44a9-8ea0-e28c5a7b86e5 None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:20:01.365760 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b4e7e8-ee65-44a9-8ea0-e28c5a7b86e5 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:20:01.366076 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-49b4e7e8-ee65-44a9-8ea0-e28c5a7b86e5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:20:01.395138 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-49b4e7e8-ee65-44a9-8ea0-e28c5a7b86e5 None None] MFA Rules not processed for user `17da521881d4420dac44868b53c55c0f`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 22:20:01.396366 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-49b4e7e8-ee65-44a9-8ea0-e28c5a7b86e5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:20:01.409424 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:20:01.469994 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1159/2312] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jul 13 22:20:01 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 13 22:20:01.474457 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205c3022-2a94-4bb9-9c28-f3c376e1b0e3 admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:20:01.516467 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:INSERT=4,keystone:SELECT=96 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:20:01.539954 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-205c3022-2a94-4bb9-9c28-f3c376e1b0e3 admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:20:01.545000 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c277233-58f7-44b8-aacd-f0f7902bc224 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:20:01.545672 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c277233-58f7-44b8-aacd-f0f7902bc224 admin admin] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:20:01.545947 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c277233-58f7-44b8-aacd-f0f7902bc224 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:20:01.546220 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8c277233-58f7-44b8-aacd-f0f7902bc224 admin admin] PATH_INFO: `/v3/projects` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:20:01.546937 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c277233-58f7-44b8-aacd-f0f7902bc224 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:20:01.549112 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8c277233-58f7-44b8-aacd-f0f7902bc224 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:20:01.578062 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1154/2313] 10.222.0.52 () {60 vars in 1152 bytes} [Wed Jul 13 22:20:01 2022] POST /identity/v3/projects => generated 387 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 13 22:20:01.582514 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b4e7e8-ee65-44a9-8ea0-e28c5a7b86e5 None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:20:01.594297 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-49b4e7e8-ee65-44a9-8ea0-e28c5a7b86e5 None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:20:01.650653 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef3e4b0-acd4-4724-9144-7be328234c6a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:20:01.651491 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef3e4b0-acd4-4724-9144-7be328234c6a admin admin] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:20:01.651592 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef3e4b0-acd4-4724-9144-7be328234c6a admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:20:01.651874 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bef3e4b0-acd4-4724-9144-7be328234c6a admin admin] PATH_INFO: `/v3/users` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:20:01.652539 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef3e4b0-acd4-4724-9144-7be328234c6a admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:20:01.654472 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bef3e4b0-acd4-4724-9144-7be328234c6a admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:20:01.678562 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1160/2314] 10.222.0.52 () {60 vars in 1143 bytes} [Wed Jul 13 22:20:01 2022] POST /identity/v3/users => generated 300 bytes in 95 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 22:20:01.681152 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c277233-58f7-44b8-aacd-f0f7902bc224 admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:20:01.691695 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8c277233-58f7-44b8-aacd-f0f7902bc224 admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:20:01.696748 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9543e79e-4ffc-4d44-be00-2ce87f2ee084 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:20:01.697420 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9543e79e-4ffc-4d44-be00-2ce87f2ee084 admin admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:20:01.697703 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9543e79e-4ffc-4d44-be00-2ce87f2ee084 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:20:01.697977 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9543e79e-4ffc-4d44-be00-2ce87f2ee084 admin admin] PATH_INFO: `/v3/roles` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:20:01.698747 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9543e79e-4ffc-4d44-be00-2ce87f2ee084 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:20:01.700368 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9543e79e-4ffc-4d44-be00-2ce87f2ee084 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:20:01.711467 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1155/2315] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jul 13 22:20:01 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 22:20:01.715874 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef3e4b0-acd4-4724-9144-7be328234c6a admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:20:01.726362 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bef3e4b0-acd4-4724-9144-7be328234c6a admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:20:01.731551 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52728846-0eac-488b-92a8-6c658cb48c71 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:20:01.732444 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52728846-0eac-488b-92a8-6c658cb48c71 admin admin] REQUEST_METHOD: `PUT` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:20:01.732872 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52728846-0eac-488b-92a8-6c658cb48c71 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:20:01.733137 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-52728846-0eac-488b-92a8-6c658cb48c71 admin admin] PATH_INFO: `/v3/projects/d8dec914eab94cfa9320b8d8917e0627/users/29b2579e16e749a59294a5c720697ddc/roles/44efba2579f342df8a0789c59d163ec3` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:20:01.751357 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52728846-0eac-488b-92a8-6c658cb48c71 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d8dec914eab94cfa9320b8d8917e0627, user_id=29b2579e16e749a59294a5c720697ddc, role_id=44efba2579f342df8a0789c59d163ec3)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:20:01.752679 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-52728846-0eac-488b-92a8-6c658cb48c71 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:20:01.772678 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1161/2316] 10.222.0.52 () {60 vars in 1482 bytes} [Wed Jul 13 22:20:01 2022] PUT /identity/v3/projects/d8dec914eab94cfa9320b8d8917e0627/users/29b2579e16e749a59294a5c720697ddc/roles/44efba2579f342df8a0789c59d163ec3 => generated 0 bytes in 57 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 22:20:01.776432 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9543e79e-4ffc-4d44-be00-2ce87f2ee084 admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:20:01.788142 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9543e79e-4ffc-4d44-be00-2ce87f2ee084 admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:20:01.854000 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d78a659d-dc2e-46ad-b62f-199de9a690fc admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:20:01.854712 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78a659d-dc2e-46ad-b62f-199de9a690fc admin admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:20:01.854992 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78a659d-dc2e-46ad-b62f-199de9a690fc admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:20:01.855268 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d78a659d-dc2e-46ad-b62f-199de9a690fc admin admin] PATH_INFO: `/v3/roles` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:20:01.856028 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78a659d-dc2e-46ad-b62f-199de9a690fc admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:20:01.857625 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d78a659d-dc2e-46ad-b62f-199de9a690fc admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:20:01.867643 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1156/2317] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jul 13 22:20:01 2022] GET /identity/v3/roles => generated 1420 bytes in 92 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 22:20:01.872103 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52728846-0eac-488b-92a8-6c658cb48c71 admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:20:01.883197 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-52728846-0eac-488b-92a8-6c658cb48c71 admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:20:01.888162 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ea2dac-91d1-49bf-81fc-bc9b0e177339 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:20:01.889064 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ea2dac-91d1-49bf-81fc-bc9b0e177339 admin admin] REQUEST_METHOD: `PUT` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:20:01.889345 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ea2dac-91d1-49bf-81fc-bc9b0e177339 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:20:01.889620 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-95ea2dac-91d1-49bf-81fc-bc9b0e177339 admin admin] PATH_INFO: `/v3/projects/d8dec914eab94cfa9320b8d8917e0627/users/29b2579e16e749a59294a5c720697ddc/roles/44efba2579f342df8a0789c59d163ec3` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:20:01.891886 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ea2dac-91d1-49bf-81fc-bc9b0e177339 admin admin] RBAC: Authorizing `identity:create_grant(project_id=d8dec914eab94cfa9320b8d8917e0627, user_id=29b2579e16e749a59294a5c720697ddc, role_id=44efba2579f342df8a0789c59d163ec3)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:20:01.894571 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-95ea2dac-91d1-49bf-81fc-bc9b0e177339 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:20:01.906132 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1162/2318] 10.222.0.52 () {60 vars in 1482 bytes} [Wed Jul 13 22:20:01 2022] PUT /identity/v3/projects/d8dec914eab94cfa9320b8d8917e0627/users/29b2579e16e749a59294a5c720697ddc/roles/44efba2579f342df8a0789c59d163ec3 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 22:20:01.917905 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01d7ea4-4cb3-41ae-bbc5-73afe9086f7b None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:20:01.918164 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01d7ea4-4cb3-41ae-bbc5-73afe9086f7b None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:20:01.918868 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e01d7ea4-4cb3-41ae-bbc5-73afe9086f7b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:20:01.947547 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-e01d7ea4-4cb3-41ae-bbc5-73afe9086f7b None None] MFA Rules not processed for user `29b2579e16e749a59294a5c720697ddc`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 22:20:01.948824 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-e01d7ea4-4cb3-41ae-bbc5-73afe9086f7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:20:02.022851 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1157/2319] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jul 13 22:20:01 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 106 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 22:20:02.034195 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ea2dac-91d1-49bf-81fc-bc9b0e177339 admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:20:02.036629 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-95ea2dac-91d1-49bf-81fc-bc9b0e177339 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:20:02.102901 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-95ea2dac-91d1-49bf-81fc-bc9b0e177339 admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:20:02.106948 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1d9bc3-545f-4f55-b9ce-35baf91473b1 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc77e1ea1ada4700b809beb1c980c5e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417e3e2f1333481795a089bef2e24bea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:20:02.107687 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1d9bc3-545f-4f55-b9ce-35baf91473b1 service cinder] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:20:02.108069 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1d9bc3-545f-4f55-b9ce-35baf91473b1 service cinder] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:20:02.108349 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db1d9bc3-545f-4f55-b9ce-35baf91473b1 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:20:02.119383 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db1d9bc3-545f-4f55-b9ce-35baf91473b1 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:20:02.120851 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-db1d9bc3-545f-4f55-b9ce-35baf91473b1 service cinder] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:20:02.130971 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1163/2320] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jul 13 22:20:02 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 22:20:02.661130 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01d7ea4-4cb3-41ae-bbc5-73afe9086f7b None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:20:02.662666 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-e01d7ea4-4cb3-41ae-bbc5-73afe9086f7b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:20:02.676318 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e01d7ea4-4cb3-41ae-bbc5-73afe9086f7b None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:20:02.679920 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d18f715-830d-47de-ab25-5fd49b47bb88 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc77e1ea1ada4700b809beb1c980c5e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417e3e2f1333481795a089bef2e24bea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:20:02.680579 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d18f715-830d-47de-ab25-5fd49b47bb88 service cinder] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:20:02.680861 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d18f715-830d-47de-ab25-5fd49b47bb88 service cinder] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:20:02.681135 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0d18f715-830d-47de-ab25-5fd49b47bb88 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:20:02.714477 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=99,keystone:INSERT=1 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:20:02.745744 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d18f715-830d-47de-ab25-5fd49b47bb88 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:20:02.747231 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0d18f715-830d-47de-ab25-5fd49b47bb88 service cinder] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:20:02.757292 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1158/2321] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jul 13 22:20:02 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 97 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 22:20:06.826733 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1d9bc3-545f-4f55-b9ce-35baf91473b1 service cinder] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:20:06.838242 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:20:06.847698 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db1d9bc3-545f-4f55-b9ce-35baf91473b1 service cinder] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:20:06.851941 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1bd98d31-f675-4985-85a2-7f72792a2e24 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc77e1ea1ada4700b809beb1c980c5e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417e3e2f1333481795a089bef2e24bea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:20:06.852584 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd98d31-f675-4985-85a2-7f72792a2e24 service cinder] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:20:06.852860 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd98d31-f675-4985-85a2-7f72792a2e24 service cinder] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:20:06.853131 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1bd98d31-f675-4985-85a2-7f72792a2e24 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:20:06.925746 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd98d31-f675-4985-85a2-7f72792a2e24 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:20:06.927228 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1bd98d31-f675-4985-85a2-7f72792a2e24 service cinder] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:20:06.936417 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1164/2322] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jul 13 22:20:06 2022] GET /identity/v3/auth/tokens => generated 3254 bytes in 111 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 22:20:09.159548 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d18f715-830d-47de-ab25-5fd49b47bb88 service cinder] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:20:09.301136 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0d18f715-830d-47de-ab25-5fd49b47bb88 service cinder] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:20:09.305676 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08cf416-03f8-430a-9677-d14833f3fc08 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:20:09.306582 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08cf416-03f8-430a-9677-d14833f3fc08 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:20:09.306860 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08cf416-03f8-430a-9677-d14833f3fc08 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:20:09.307130 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d08cf416-03f8-430a-9677-d14833f3fc08 admin admin] PATH_INFO: `/v3/users/019717606a664ad4ab4277ac219e875c` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:20:09.308331 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08cf416-03f8-430a-9677-d14833f3fc08 admin admin] RBAC: Authorizing `identity:delete_user(user_id=019717606a664ad4ab4277ac219e875c)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:20:09.310379 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d08cf416-03f8-430a-9677-d14833f3fc08 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:20:09.362391 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-d08cf416-03f8-430a-9677-d14833f3fc08 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '019717606a664ad4ab4277ac219e875c'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:20:09.371325 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-d08cf416-03f8-430a-9677-d14833f3fc08 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '019717606a664ad4ab4277ac219e875c'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:20:09.385439 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-d08cf416-03f8-430a-9677-d14833f3fc08 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '019717606a664ad4ab4277ac219e875c'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:20:09.386080 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-d08cf416-03f8-430a-9677-d14833f3fc08 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '019717606a664ad4ab4277ac219e875c'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:20:09.386725 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-d08cf416-03f8-430a-9677-d14833f3fc08 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '019717606a664ad4ab4277ac219e875c'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:20:09.400427 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1159/2323] 10.222.0.52 () {58 vars in 1223 bytes} [Wed Jul 13 22:20:09 2022] DELETE /identity/v3/users/019717606a664ad4ab4277ac219e875c => generated 0 bytes in 242 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 22:20:09.405316 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43cba68-72e2-476a-afad-1d863b07fb0f None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:20:09.405846 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43cba68-72e2-476a-afad-1d863b07fb0f None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:20:09.405955 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c43cba68-72e2-476a-afad-1d863b07fb0f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:20:09.467521 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-c43cba68-72e2-476a-afad-1d863b07fb0f None None] MFA Rules not processed for user `cc99c93da5074cf5a48bc2aa9a92c5e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 22:20:09.468878 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-c43cba68-72e2-476a-afad-1d863b07fb0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:20:09.538451 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1165/2324] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jul 13 22:20:09 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 134 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 22:20:09.548769 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08cf416-03f8-430a-9677-d14833f3fc08 admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:20:09.550395 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-d08cf416-03f8-430a-9677-d14833f3fc08 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:20:09.560218 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:20:09.619685 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d08cf416-03f8-430a-9677-d14833f3fc08 admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:20:09.623539 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a836c8d-a059-4ebd-a94c-b0923f73a7a9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '847ec9a1bd4d4dec804318a5c845895a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417e3e2f1333481795a089bef2e24bea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:20:09.624193 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a836c8d-a059-4ebd-a94c-b0923f73a7a9 service neutron] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:20:09.624472 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a836c8d-a059-4ebd-a94c-b0923f73a7a9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:20:09.624752 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a836c8d-a059-4ebd-a94c-b0923f73a7a9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:20:09.634603 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a836c8d-a059-4ebd-a94c-b0923f73a7a9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:20:09.636095 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1a836c8d-a059-4ebd-a94c-b0923f73a7a9 service neutron] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:20:09.645858 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1160/2325] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jul 13 22:20:09 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 22:20:09.921245 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43cba68-72e2-476a-afad-1d863b07fb0f None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:20:09.923160 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-c43cba68-72e2-476a-afad-1d863b07fb0f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:20:09.940291 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c43cba68-72e2-476a-afad-1d863b07fb0f None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:20:09.946252 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c8e8d2-684f-4587-899f-469c297b16cf admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:20:09.947178 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c8e8d2-684f-4587-899f-469c297b16cf admin admin] REQUEST_METHOD: `DELETE` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:20:09.947453 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c8e8d2-684f-4587-899f-469c297b16cf admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:20:09.947733 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64c8e8d2-684f-4587-899f-469c297b16cf admin admin] PATH_INFO: `/v3/projects/c2f2f47e8fae45898130b3410f6cc5ca` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:20:09.949016 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c8e8d2-684f-4587-899f-469c297b16cf admin admin] RBAC: Authorizing `identity:delete_project(project_id=c2f2f47e8fae45898130b3410f6cc5ca)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:20:09.950984 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64c8e8d2-684f-4587-899f-469c297b16cf admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:20:10.000085 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-64c8e8d2-684f-4587-899f-469c297b16cf admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'c2f2f47e8fae45898130b3410f6cc5ca', 'request_id': 'req-64c8e8d2-684f-4587-899f-469c297b16cf'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:20:10.001989 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=96,keystone:DELETE=4 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:20:10.007064 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-64c8e8d2-684f-4587-899f-469c297b16cf admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'c2f2f47e8fae45898130b3410f6cc5ca', 'request_id': 'req-64c8e8d2-684f-4587-899f-469c297b16cf'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:20:10.014103 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-64c8e8d2-684f-4587-899f-469c297b16cf admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'c2f2f47e8fae45898130b3410f6cc5ca', 'request_id': 'req-64c8e8d2-684f-4587-899f-469c297b16cf'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:20:10.031119 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-64c8e8d2-684f-4587-899f-469c297b16cf admin admin] The token cache is being invalidate because project c2f2f47e8fae45898130b3410f6cc5ca was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80381) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 22:20:10.031483 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-64c8e8d2-684f-4587-899f-469c297b16cf admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:20:10.032161 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-64c8e8d2-684f-4587-899f-469c297b16cf admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:20:10.033893 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1166/2326] 10.222.0.52 () {58 vars in 1232 bytes} [Wed Jul 13 22:20:09 2022] DELETE /identity/v3/projects/c2f2f47e8fae45898130b3410f6cc5ca => generated 0 bytes in 113 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 22:20:10.040445 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a836c8d-a059-4ebd-a94c-b0923f73a7a9 service neutron] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:20:10.055310 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a836c8d-a059-4ebd-a94c-b0923f73a7a9 service neutron] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:20:10.119103 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56e91ea-f082-4176-b979-534183cba482 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:20:10.119992 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56e91ea-f082-4176-b979-534183cba482 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:20:10.120271 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56e91ea-f082-4176-b979-534183cba482 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:20:10.120563 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f56e91ea-f082-4176-b979-534183cba482 admin admin] PATH_INFO: `/v3/users/17da521881d4420dac44868b53c55c0f` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:20:10.121797 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f56e91ea-f082-4176-b979-534183cba482 admin admin] RBAC: Authorizing `identity:delete_user(user_id=17da521881d4420dac44868b53c55c0f)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:20:10.123764 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f56e91ea-f082-4176-b979-534183cba482 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:20:10.173156 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-f56e91ea-f082-4176-b979-534183cba482 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '17da521881d4420dac44868b53c55c0f'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:20:10.181599 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-f56e91ea-f082-4176-b979-534183cba482 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '17da521881d4420dac44868b53c55c0f'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:20:10.193309 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-f56e91ea-f082-4176-b979-534183cba482 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '17da521881d4420dac44868b53c55c0f'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:20:10.194103 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-f56e91ea-f082-4176-b979-534183cba482 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '17da521881d4420dac44868b53c55c0f'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:20:10.194721 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-f56e91ea-f082-4176-b979-534183cba482 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '17da521881d4420dac44868b53c55c0f'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:20:10.208278 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1161/2327] 10.222.0.52 () {58 vars in 1223 bytes} [Wed Jul 13 22:20:10 2022] DELETE /identity/v3/users/17da521881d4420dac44868b53c55c0f => generated 0 bytes in 169 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 22:20:10.509583 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c8e8d2-684f-4587-899f-469c297b16cf admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:20:10.511141 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-64c8e8d2-684f-4587-899f-469c297b16cf admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:20:10.524701 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64c8e8d2-684f-4587-899f-469c297b16cf admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:20:10.585838 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db22bea-cbb4-46c3-9f97-fe73939e4df3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:20:10.586748 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db22bea-cbb4-46c3-9f97-fe73939e4df3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:20:10.587024 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db22bea-cbb4-46c3-9f97-fe73939e4df3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:20:10.587293 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7db22bea-cbb4-46c3-9f97-fe73939e4df3 admin admin] PATH_INFO: `/v3/projects/a38b9915ebfb4968939d8d9e87d78638` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:20:10.588583 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db22bea-cbb4-46c3-9f97-fe73939e4df3 admin admin] RBAC: Authorizing `identity:delete_project(project_id=a38b9915ebfb4968939d8d9e87d78638)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:20:10.590577 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7db22bea-cbb4-46c3-9f97-fe73939e4df3 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:20:10.636838 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-7db22bea-cbb4-46c3-9f97-fe73939e4df3 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'a38b9915ebfb4968939d8d9e87d78638', 'request_id': 'req-7db22bea-cbb4-46c3-9f97-fe73939e4df3'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:20:10.643101 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-7db22bea-cbb4-46c3-9f97-fe73939e4df3 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'a38b9915ebfb4968939d8d9e87d78638', 'request_id': 'req-7db22bea-cbb4-46c3-9f97-fe73939e4df3'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:20:10.648869 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-7db22bea-cbb4-46c3-9f97-fe73939e4df3 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'a38b9915ebfb4968939d8d9e87d78638', 'request_id': 'req-7db22bea-cbb4-46c3-9f97-fe73939e4df3'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:20:10.665200 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-7db22bea-cbb4-46c3-9f97-fe73939e4df3 admin admin] The token cache is being invalidate because project a38b9915ebfb4968939d8d9e87d78638 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80381) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 22:20:10.665581 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-7db22bea-cbb4-46c3-9f97-fe73939e4df3 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:20:10.666765 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-7db22bea-cbb4-46c3-9f97-fe73939e4df3 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:20:10.669019 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1167/2328] 10.222.0.52 () {58 vars in 1232 bytes} [Wed Jul 13 22:20:10 2022] DELETE /identity/v3/projects/a38b9915ebfb4968939d8d9e87d78638 => generated 0 bytes in 160 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 22:20:10.674381 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56e91ea-f082-4176-b979-534183cba482 admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:20:10.675982 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-f56e91ea-f082-4176-b979-534183cba482 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:20:10.689724 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f56e91ea-f082-4176-b979-534183cba482 admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:20:10.710122 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:20:10.748813 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-64ad7abb-0cda-49a9-99f6-06d149aebe4c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:20:10.749694 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ad7abb-0cda-49a9-99f6-06d149aebe4c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:20:10.750051 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ad7abb-0cda-49a9-99f6-06d149aebe4c admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:20:10.750367 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-64ad7abb-0cda-49a9-99f6-06d149aebe4c admin admin] PATH_INFO: `/v3/users/29b2579e16e749a59294a5c720697ddc` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:20:10.751658 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ad7abb-0cda-49a9-99f6-06d149aebe4c admin admin] RBAC: Authorizing `identity:delete_user(user_id=29b2579e16e749a59294a5c720697ddc)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:20:10.753610 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-64ad7abb-0cda-49a9-99f6-06d149aebe4c admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:20:10.814529 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-64ad7abb-0cda-49a9-99f6-06d149aebe4c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '29b2579e16e749a59294a5c720697ddc'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:20:10.822738 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-64ad7abb-0cda-49a9-99f6-06d149aebe4c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '29b2579e16e749a59294a5c720697ddc'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:20:10.835659 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-64ad7abb-0cda-49a9-99f6-06d149aebe4c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '29b2579e16e749a59294a5c720697ddc'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:20:10.836374 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-64ad7abb-0cda-49a9-99f6-06d149aebe4c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '29b2579e16e749a59294a5c720697ddc'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:20:10.837063 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-64ad7abb-0cda-49a9-99f6-06d149aebe4c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '29b2579e16e749a59294a5c720697ddc'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:20:10.849615 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1162/2329] 10.222.0.52 () {58 vars in 1223 bytes} [Wed Jul 13 22:20:10 2022] DELETE /identity/v3/users/29b2579e16e749a59294a5c720697ddc => generated 0 bytes in 176 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 22:20:11.123741 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db22bea-cbb4-46c3-9f97-fe73939e4df3 admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:20:11.125423 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-7db22bea-cbb4-46c3-9f97-fe73939e4df3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:20:11.138718 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7db22bea-cbb4-46c3-9f97-fe73939e4df3 admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:20:11.197124 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1d69727-55e4-4346-ac2e-6a77178293a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:20:11.198000 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d69727-55e4-4346-ac2e-6a77178293a4 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:20:11.198275 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d69727-55e4-4346-ac2e-6a77178293a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:20:11.198606 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1d69727-55e4-4346-ac2e-6a77178293a4 admin admin] PATH_INFO: `/v3/projects/d8dec914eab94cfa9320b8d8917e0627` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:20:11.199775 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d69727-55e4-4346-ac2e-6a77178293a4 admin admin] RBAC: Authorizing `identity:delete_project(project_id=d8dec914eab94cfa9320b8d8917e0627)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:20:11.201676 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1d69727-55e4-4346-ac2e-6a77178293a4 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:20:11.212880 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=88,keystone:DELETE=10,keystone:INSERT=2 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:20:11.249364 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-a1d69727-55e4-4346-ac2e-6a77178293a4 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'd8dec914eab94cfa9320b8d8917e0627', 'request_id': 'req-a1d69727-55e4-4346-ac2e-6a77178293a4'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:20:11.255332 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-a1d69727-55e4-4346-ac2e-6a77178293a4 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'd8dec914eab94cfa9320b8d8917e0627', 'request_id': 'req-a1d69727-55e4-4346-ac2e-6a77178293a4'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:20:11.261410 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-a1d69727-55e4-4346-ac2e-6a77178293a4 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'd8dec914eab94cfa9320b8d8917e0627', 'request_id': 'req-a1d69727-55e4-4346-ac2e-6a77178293a4'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:20:11.278746 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-a1d69727-55e4-4346-ac2e-6a77178293a4 admin admin] The token cache is being invalidate because project d8dec914eab94cfa9320b8d8917e0627 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80381) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 22:20:11.279029 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-a1d69727-55e4-4346-ac2e-6a77178293a4 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:20:11.279660 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-a1d69727-55e4-4346-ac2e-6a77178293a4 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:20:11.281233 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1168/2330] 10.222.0.52 () {58 vars in 1232 bytes} [Wed Jul 13 22:20:11 2022] DELETE /identity/v3/projects/d8dec914eab94cfa9320b8d8917e0627 => generated 0 bytes in 158 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 22:20:11.287606 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c735a806-76ed-423f-8d63-6bec5be91058 None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:20:11.288082 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c735a806-76ed-423f-8d63-6bec5be91058 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:20:11.288202 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c735a806-76ed-423f-8d63-6bec5be91058 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:20:11.337402 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-c735a806-76ed-423f-8d63-6bec5be91058 None None] MFA Rules not processed for user `cc99c93da5074cf5a48bc2aa9a92c5e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 22:20:11.338631 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-c735a806-76ed-423f-8d63-6bec5be91058 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:20:11.399778 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1163/2331] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jul 13 22:20:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 113 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 13 22:20:11.411618 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e7c14a-0890-4dde-9081-beab6c118146 None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:20:11.411926 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e7c14a-0890-4dde-9081-beab6c118146 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:20:11.412203 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3e7c14a-0890-4dde-9081-beab6c118146 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:20:11.433523 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-e3e7c14a-0890-4dde-9081-beab6c118146 None None] MFA Rules not processed for user `cc99c93da5074cf5a48bc2aa9a92c5e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 22:20:11.434790 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-e3e7c14a-0890-4dde-9081-beab6c118146 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:20:11.443864 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1169/2332] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jul 13 22:20:11 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 33 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 22:20:11.448705 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c735a806-76ed-423f-8d63-6bec5be91058 None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:20:11.459243 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c735a806-76ed-423f-8d63-6bec5be91058 None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:20:11.464666 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd94ad6-4f0c-4abd-b524-2cea79a3759a admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:20:11.465366 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd94ad6-4f0c-4abd-b524-2cea79a3759a admin admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:20:11.465654 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd94ad6-4f0c-4abd-b524-2cea79a3759a admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:20:11.465937 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1fd94ad6-4f0c-4abd-b524-2cea79a3759a admin admin] PATH_INFO: `/v3/domains` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:20:11.466709 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd94ad6-4f0c-4abd-b524-2cea79a3759a admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 22:20:11.467053 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd94ad6-4f0c-4abd-b524-2cea79a3759a admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:20:11.468730 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1fd94ad6-4f0c-4abd-b524-2cea79a3759a admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:20:11.480837 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1164/2333] 10.222.0.52 () {58 vars in 1152 bytes} [Wed Jul 13 22:20:11 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 33 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 22:20:11.485173 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e7c14a-0890-4dde-9081-beab6c118146 None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:20:11.495822 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3e7c14a-0890-4dde-9081-beab6c118146 None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:20:11.503002 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b0601c-2420-4ca2-90a7-bda92070c29c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:20:11.503604 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b0601c-2420-4ca2-90a7-bda92070c29c admin admin] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:20:11.503880 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b0601c-2420-4ca2-90a7-bda92070c29c admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:20:11.504145 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79b0601c-2420-4ca2-90a7-bda92070c29c admin admin] PATH_INFO: `/v3/projects` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:20:11.506433 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b0601c-2420-4ca2-90a7-bda92070c29c admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:20:11.508358 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79b0601c-2420-4ca2-90a7-bda92070c29c admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:20:11.545631 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1170/2334] 10.222.0.52 () {60 vars in 1152 bytes} [Wed Jul 13 22:20:11 2022] POST /identity/v3/projects => generated 393 bytes in 61 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 22:20:11.552573 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd94ad6-4f0c-4abd-b524-2cea79a3759a admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:20:11.563014 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:20:11.567077 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1fd94ad6-4f0c-4abd-b524-2cea79a3759a admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:20:11.632367 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03cdd623-7e5f-4938-8466-55c480954935 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:20:11.633051 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cdd623-7e5f-4938-8466-55c480954935 admin admin] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:20:11.633331 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cdd623-7e5f-4938-8466-55c480954935 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:20:11.633605 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-03cdd623-7e5f-4938-8466-55c480954935 admin admin] PATH_INFO: `/v3/users` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:20:11.634336 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03cdd623-7e5f-4938-8466-55c480954935 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:20:11.636596 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-03cdd623-7e5f-4938-8466-55c480954935 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:20:11.661292 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1165/2335] 10.222.0.52 () {60 vars in 1143 bytes} [Wed Jul 13 22:20:11 2022] POST /identity/v3/users => generated 297 bytes in 107 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 22:20:11.662608 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b0601c-2420-4ca2-90a7-bda92070c29c admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:20:11.674426 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79b0601c-2420-4ca2-90a7-bda92070c29c admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:20:11.680118 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138bb141-49bb-4bb2-a335-5790a0800211 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:20:11.680798 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138bb141-49bb-4bb2-a335-5790a0800211 admin admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:20:11.681096 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138bb141-49bb-4bb2-a335-5790a0800211 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:20:11.681342 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-138bb141-49bb-4bb2-a335-5790a0800211 admin admin] PATH_INFO: `/v3/roles` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:20:11.682074 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138bb141-49bb-4bb2-a335-5790a0800211 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:20:11.683733 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-138bb141-49bb-4bb2-a335-5790a0800211 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:20:11.697212 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1171/2336] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jul 13 22:20:11 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 13 22:20:11.701384 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03cdd623-7e5f-4938-8466-55c480954935 admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:20:11.711916 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-03cdd623-7e5f-4938-8466-55c480954935 admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:20:11.716914 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-703e4cc9-0d64-40a9-a813-84ba9508c5a3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:20:11.717658 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703e4cc9-0d64-40a9-a813-84ba9508c5a3 admin admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:20:11.718007 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703e4cc9-0d64-40a9-a813-84ba9508c5a3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:20:11.718343 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-703e4cc9-0d64-40a9-a813-84ba9508c5a3 admin admin] PATH_INFO: `/v3/roles` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:20:11.719548 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703e4cc9-0d64-40a9-a813-84ba9508c5a3 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:20:11.721261 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-703e4cc9-0d64-40a9-a813-84ba9508c5a3 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:20:11.732190 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1166/2337] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jul 13 22:20:11 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 22:20:11.736427 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138bb141-49bb-4bb2-a335-5790a0800211 admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:20:11.757893 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-138bb141-49bb-4bb2-a335-5790a0800211 admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:20:11.757893 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4b1966-722a-4cd4-8b0b-b32f9af5091c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:20:11.757893 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4b1966-722a-4cd4-8b0b-b32f9af5091c admin admin] REQUEST_METHOD: `PUT` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:20:11.757893 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4b1966-722a-4cd4-8b0b-b32f9af5091c admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:20:11.758237 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9f4b1966-722a-4cd4-8b0b-b32f9af5091c admin admin] PATH_INFO: `/v3/projects/18f2482f8cd04bb78f9d95ac3388a35c/users/3fec9eb755f84b8487f273cf83b519d0/roles/2a5e86976394498abb6310697cdb3f8b` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:20:11.774204 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f4b1966-722a-4cd4-8b0b-b32f9af5091c admin admin] RBAC: Authorizing `identity:create_grant(project_id=18f2482f8cd04bb78f9d95ac3388a35c, user_id=3fec9eb755f84b8487f273cf83b519d0, role_id=2a5e86976394498abb6310697cdb3f8b)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:20:11.776972 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-9f4b1966-722a-4cd4-8b0b-b32f9af5091c admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:20:11.791968 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1172/2338] 10.222.0.52 () {60 vars in 1482 bytes} [Wed Jul 13 22:20:11 2022] PUT /identity/v3/projects/18f2482f8cd04bb78f9d95ac3388a35c/users/3fec9eb755f84b8487f273cf83b519d0/roles/2a5e86976394498abb6310697cdb3f8b => generated 0 bytes in 56 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 22:20:11.804635 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb8c3c1-fcdc-46ac-810d-2f404db9ac4e None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:20:11.804894 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb8c3c1-fcdc-46ac-810d-2f404db9ac4e None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:20:11.805191 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1cb8c3c1-fcdc-46ac-810d-2f404db9ac4e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:20:11.834219 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-1cb8c3c1-fcdc-46ac-810d-2f404db9ac4e None None] MFA Rules not processed for user `3fec9eb755f84b8487f273cf83b519d0`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 22:20:11.835464 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-1cb8c3c1-fcdc-46ac-810d-2f404db9ac4e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:20:11.902443 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1167/2339] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jul 13 22:20:11 2022] POST /identity/v3/auth/tokens => generated 3260 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 22:20:11.907068 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4b1966-722a-4cd4-8b0b-b32f9af5091c admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:20:11.974236 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9f4b1966-722a-4cd4-8b0b-b32f9af5091c admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:20:11.979094 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1260dcd-d1a2-480c-91c5-5b880458e220 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:20:11.979857 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1260dcd-d1a2-480c-91c5-5b880458e220 admin admin] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:20:11.980133 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1260dcd-d1a2-480c-91c5-5b880458e220 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:20:11.980399 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e1260dcd-d1a2-480c-91c5-5b880458e220 admin admin] PATH_INFO: `/v3/projects` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:20:11.981093 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1260dcd-d1a2-480c-91c5-5b880458e220 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:20:11.983327 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e1260dcd-d1a2-480c-91c5-5b880458e220 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:20:12.014100 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1173/2340] 10.222.0.52 () {60 vars in 1152 bytes} [Wed Jul 13 22:20:11 2022] POST /identity/v3/projects => generated 393 bytes in 104 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 22:20:12.015865 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb8c3c1-fcdc-46ac-810d-2f404db9ac4e None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:20:12.027173 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1cb8c3c1-fcdc-46ac-810d-2f404db9ac4e None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:20:12.088042 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97025d3-d455-42e7-b0f4-8b8dd8eff8e8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:20:12.088799 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97025d3-d455-42e7-b0f4-8b8dd8eff8e8 admin admin] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:20:12.089139 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97025d3-d455-42e7-b0f4-8b8dd8eff8e8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:20:12.089415 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c97025d3-d455-42e7-b0f4-8b8dd8eff8e8 admin admin] PATH_INFO: `/v3/users` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:20:12.090929 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97025d3-d455-42e7-b0f4-8b8dd8eff8e8 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:20:12.092283 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c97025d3-d455-42e7-b0f4-8b8dd8eff8e8 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:20:12.117092 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1168/2341] 10.222.0.52 () {60 vars in 1143 bytes} [Wed Jul 13 22:20:12 2022] POST /identity/v3/users => generated 303 bytes in 102 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 13 22:20:12.121201 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1260dcd-d1a2-480c-91c5-5b880458e220 admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:20:12.131746 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e1260dcd-d1a2-480c-91c5-5b880458e220 admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:20:12.136865 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c5d258-7b74-4f68-9ab9-c5043b2f8ba8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:20:12.136980 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c5d258-7b74-4f68-9ab9-c5043b2f8ba8 admin admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:20:12.137212 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c5d258-7b74-4f68-9ab9-c5043b2f8ba8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:20:12.137525 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b4c5d258-7b74-4f68-9ab9-c5043b2f8ba8 admin admin] PATH_INFO: `/v3/roles` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:20:12.138638 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c5d258-7b74-4f68-9ab9-c5043b2f8ba8 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:20:12.140639 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b4c5d258-7b74-4f68-9ab9-c5043b2f8ba8 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:20:12.151872 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1174/2342] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jul 13 22:20:12 2022] GET /identity/v3/roles => generated 1420 bytes in 31 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 22:20:12.156302 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97025d3-d455-42e7-b0f4-8b8dd8eff8e8 admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:20:12.161750 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=94,keystone:INSERT=6 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:20:12.168420 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c97025d3-d455-42e7-b0f4-8b8dd8eff8e8 admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:20:12.176864 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e37f33-871f-4000-a6e4-6e2cd09959f8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:20:12.177923 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e37f33-871f-4000-a6e4-6e2cd09959f8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:20:12.178682 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e37f33-871f-4000-a6e4-6e2cd09959f8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:20:12.178973 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1e37f33-871f-4000-a6e4-6e2cd09959f8 admin admin] PATH_INFO: `/v3/projects/34078dc8acc54dcfa1ae5f3a0c5fbf20/users/3c027f34670e4b9ca3568baed7786296/roles/44efba2579f342df8a0789c59d163ec3` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:20:12.197296 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e37f33-871f-4000-a6e4-6e2cd09959f8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=34078dc8acc54dcfa1ae5f3a0c5fbf20, user_id=3c027f34670e4b9ca3568baed7786296, role_id=44efba2579f342df8a0789c59d163ec3)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:20:12.200152 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1e37f33-871f-4000-a6e4-6e2cd09959f8 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:20:12.216296 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1169/2343] 10.222.0.52 () {60 vars in 1482 bytes} [Wed Jul 13 22:20:12 2022] PUT /identity/v3/projects/34078dc8acc54dcfa1ae5f3a0c5fbf20/users/3c027f34670e4b9ca3568baed7786296/roles/44efba2579f342df8a0789c59d163ec3 => generated 0 bytes in 61 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 13 22:20:12.220741 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c5d258-7b74-4f68-9ab9-c5043b2f8ba8 admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:20:12.232385 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b4c5d258-7b74-4f68-9ab9-c5043b2f8ba8 admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:20:12.243220 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:20:12.299281 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc2a878-4685-487f-8e84-a43022fedc43 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:20:12.299853 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc2a878-4685-487f-8e84-a43022fedc43 admin admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:20:12.300134 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc2a878-4685-487f-8e84-a43022fedc43 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:20:12.300410 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc2a878-4685-487f-8e84-a43022fedc43 admin admin] PATH_INFO: `/v3/roles` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:20:12.301150 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc2a878-4685-487f-8e84-a43022fedc43 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:20:12.302806 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc2a878-4685-487f-8e84-a43022fedc43 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:20:12.313318 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1175/2344] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jul 13 22:20:12 2022] GET /identity/v3/roles => generated 1420 bytes in 93 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 22:20:12.317617 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e37f33-871f-4000-a6e4-6e2cd09959f8 admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:20:12.330001 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1e37f33-871f-4000-a6e4-6e2cd09959f8 admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:20:12.334842 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4750b47-df6b-45b5-be86-2139096d82a4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:20:12.335775 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4750b47-df6b-45b5-be86-2139096d82a4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:20:12.336055 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4750b47-df6b-45b5-be86-2139096d82a4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:20:12.336332 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f4750b47-df6b-45b5-be86-2139096d82a4 admin admin] PATH_INFO: `/v3/projects/34078dc8acc54dcfa1ae5f3a0c5fbf20/users/3c027f34670e4b9ca3568baed7786296/roles/44efba2579f342df8a0789c59d163ec3` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:20:12.338322 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4750b47-df6b-45b5-be86-2139096d82a4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=34078dc8acc54dcfa1ae5f3a0c5fbf20, user_id=3c027f34670e4b9ca3568baed7786296, role_id=44efba2579f342df8a0789c59d163ec3)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:20:12.341021 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f4750b47-df6b-45b5-be86-2139096d82a4 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:20:12.352086 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1170/2345] 10.222.0.52 () {60 vars in 1482 bytes} [Wed Jul 13 22:20:12 2022] PUT /identity/v3/projects/34078dc8acc54dcfa1ae5f3a0c5fbf20/users/3c027f34670e4b9ca3568baed7786296/roles/44efba2579f342df8a0789c59d163ec3 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 13 22:20:12.362479 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c7db94-7ea0-4eee-882b-a25bb09256f8 None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:20:12.362806 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c7db94-7ea0-4eee-882b-a25bb09256f8 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:20:12.363085 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e6c7db94-7ea0-4eee-882b-a25bb09256f8 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:20:12.390954 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-e6c7db94-7ea0-4eee-882b-a25bb09256f8 None None] MFA Rules not processed for user `3c027f34670e4b9ca3568baed7786296`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 22:20:12.392181 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-e6c7db94-7ea0-4eee-882b-a25bb09256f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:20:12.462822 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1176/2346] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jul 13 22:20:12 2022] POST /identity/v3/auth/tokens => generated 3327 bytes in 101 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 22:20:12.473986 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4750b47-df6b-45b5-be86-2139096d82a4 admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:20:12.475924 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-f4750b47-df6b-45b5-be86-2139096d82a4 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:20:12.535210 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f4750b47-df6b-45b5-be86-2139096d82a4 admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:20:12.545433 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b917b2d-d554-4c1d-b0ed-2abeeffa2ed4 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc77e1ea1ada4700b809beb1c980c5e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417e3e2f1333481795a089bef2e24bea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:20:12.546106 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b917b2d-d554-4c1d-b0ed-2abeeffa2ed4 service cinder] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:20:12.546392 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b917b2d-d554-4c1d-b0ed-2abeeffa2ed4 service cinder] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:20:12.546797 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7b917b2d-d554-4c1d-b0ed-2abeeffa2ed4 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:20:12.559639 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b917b2d-d554-4c1d-b0ed-2abeeffa2ed4 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:20:12.561194 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7b917b2d-d554-4c1d-b0ed-2abeeffa2ed4 service cinder] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:20:12.571934 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1171/2347] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jul 13 22:20:12 2022] GET /identity/v3/auth/tokens => generated 3327 bytes in 98 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 22:20:12.623765 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c7db94-7ea0-4eee-882b-a25bb09256f8 None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:20:12.625866 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-e6c7db94-7ea0-4eee-882b-a25bb09256f8 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:20:12.641739 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e6c7db94-7ea0-4eee-882b-a25bb09256f8 None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:20:12.645546 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-47a34c26-a592-40ce-be9c-7faed90770fe service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc77e1ea1ada4700b809beb1c980c5e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417e3e2f1333481795a089bef2e24bea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:20:12.646369 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a34c26-a592-40ce-be9c-7faed90770fe service cinder] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:20:12.646722 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a34c26-a592-40ce-be9c-7faed90770fe service cinder] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:20:12.646975 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-47a34c26-a592-40ce-be9c-7faed90770fe service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:20:12.706815 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a34c26-a592-40ce-be9c-7faed90770fe service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:20:12.708317 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-47a34c26-a592-40ce-be9c-7faed90770fe service cinder] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:20:12.717426 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1177/2348] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jul 13 22:20:12 2022] GET /identity/v3/auth/tokens => generated 3260 bytes in 94 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 22:20:16.127796 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b917b2d-d554-4c1d-b0ed-2abeeffa2ed4 service cinder] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:20:16.194209 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7b917b2d-d554-4c1d-b0ed-2abeeffa2ed4 service cinder] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:20:16.199910 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e5f82e-7f40-4d8d-b205-1cfd6029753e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:20:16.200804 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e5f82e-7f40-4d8d-b205-1cfd6029753e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:20:16.201083 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e5f82e-7f40-4d8d-b205-1cfd6029753e admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:20:16.201340 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-28e5f82e-7f40-4d8d-b205-1cfd6029753e admin admin] PATH_INFO: `/v3/users/3fec9eb755f84b8487f273cf83b519d0` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:20:16.202666 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e5f82e-7f40-4d8d-b205-1cfd6029753e admin admin] RBAC: Authorizing `identity:delete_user(user_id=3fec9eb755f84b8487f273cf83b519d0)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:20:16.204727 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-28e5f82e-7f40-4d8d-b205-1cfd6029753e admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:20:16.255455 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-28e5f82e-7f40-4d8d-b205-1cfd6029753e admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3fec9eb755f84b8487f273cf83b519d0'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:20:16.263728 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-28e5f82e-7f40-4d8d-b205-1cfd6029753e admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3fec9eb755f84b8487f273cf83b519d0'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:20:16.275152 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=87,keystone:INSERT=3,keystone:DELETE=10 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:20:16.276520 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-28e5f82e-7f40-4d8d-b205-1cfd6029753e admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3fec9eb755f84b8487f273cf83b519d0'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:20:16.277292 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-28e5f82e-7f40-4d8d-b205-1cfd6029753e admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3fec9eb755f84b8487f273cf83b519d0'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:20:16.277946 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-28e5f82e-7f40-4d8d-b205-1cfd6029753e admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3fec9eb755f84b8487f273cf83b519d0'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:20:16.292510 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1172/2349] 10.222.0.52 () {58 vars in 1223 bytes} [Wed Jul 13 22:20:16 2022] DELETE /identity/v3/users/3fec9eb755f84b8487f273cf83b519d0 => generated 0 bytes in 165 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 22:20:16.298526 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c48a60b-4eda-4e9c-87ab-dd4a93b01d75 None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:20:16.299003 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c48a60b-4eda-4e9c-87ab-dd4a93b01d75 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:20:16.299096 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c48a60b-4eda-4e9c-87ab-dd4a93b01d75 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:20:16.316456 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-1c48a60b-4eda-4e9c-87ab-dd4a93b01d75 None None] MFA Rules not processed for user `cc99c93da5074cf5a48bc2aa9a92c5e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 22:20:16.317655 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-1c48a60b-4eda-4e9c-87ab-dd4a93b01d75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:20:16.342969 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=100 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:20:16.379357 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1178/2350] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jul 13 22:20:16 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 22:20:16.389313 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e5f82e-7f40-4d8d-b205-1cfd6029753e admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:20:16.390815 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-28e5f82e-7f40-4d8d-b205-1cfd6029753e admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:20:16.460886 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-28e5f82e-7f40-4d8d-b205-1cfd6029753e admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:20:16.464661 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7842ab7e-02d3-4944-b792-05b09bb407f7 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '847ec9a1bd4d4dec804318a5c845895a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417e3e2f1333481795a089bef2e24bea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:20:16.465348 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7842ab7e-02d3-4944-b792-05b09bb407f7 service neutron] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:20:16.465626 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7842ab7e-02d3-4944-b792-05b09bb407f7 service neutron] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:20:16.465870 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7842ab7e-02d3-4944-b792-05b09bb407f7 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:20:16.475641 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7842ab7e-02d3-4944-b792-05b09bb407f7 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:20:16.477090 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7842ab7e-02d3-4944-b792-05b09bb407f7 service neutron] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:20:16.488720 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1173/2351] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jul 13 22:20:16 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 100 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 22:20:16.761977 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c48a60b-4eda-4e9c-87ab-dd4a93b01d75 None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:20:16.763479 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-1c48a60b-4eda-4e9c-87ab-dd4a93b01d75 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:20:16.775775 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c48a60b-4eda-4e9c-87ab-dd4a93b01d75 None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:20:16.780504 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4cd0f9-9794-43d0-8745-3cd3251874c0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:20:16.781382 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4cd0f9-9794-43d0-8745-3cd3251874c0 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:20:16.782008 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4cd0f9-9794-43d0-8745-3cd3251874c0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:20:16.782125 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa4cd0f9-9794-43d0-8745-3cd3251874c0 admin admin] PATH_INFO: `/v3/projects/18f2482f8cd04bb78f9d95ac3388a35c` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:20:16.783470 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4cd0f9-9794-43d0-8745-3cd3251874c0 admin admin] RBAC: Authorizing `identity:delete_project(project_id=18f2482f8cd04bb78f9d95ac3388a35c)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:20:16.785438 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-aa4cd0f9-9794-43d0-8745-3cd3251874c0 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:20:16.834638 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-aa4cd0f9-9794-43d0-8745-3cd3251874c0 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '18f2482f8cd04bb78f9d95ac3388a35c', 'request_id': 'req-aa4cd0f9-9794-43d0-8745-3cd3251874c0'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:20:16.840710 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-aa4cd0f9-9794-43d0-8745-3cd3251874c0 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '18f2482f8cd04bb78f9d95ac3388a35c', 'request_id': 'req-aa4cd0f9-9794-43d0-8745-3cd3251874c0'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:20:16.846845 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-aa4cd0f9-9794-43d0-8745-3cd3251874c0 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '18f2482f8cd04bb78f9d95ac3388a35c', 'request_id': 'req-aa4cd0f9-9794-43d0-8745-3cd3251874c0'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:20:16.861213 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-aa4cd0f9-9794-43d0-8745-3cd3251874c0 admin admin] The token cache is being invalidate because project 18f2482f8cd04bb78f9d95ac3388a35c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80381) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 22:20:16.861493 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-aa4cd0f9-9794-43d0-8745-3cd3251874c0 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:20:16.862176 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-aa4cd0f9-9794-43d0-8745-3cd3251874c0 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:20:16.863837 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1179/2352] 10.222.0.52 () {58 vars in 1232 bytes} [Wed Jul 13 22:20:16 2022] DELETE /identity/v3/projects/18f2482f8cd04bb78f9d95ac3388a35c => generated 0 bytes in 103 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 22:20:16.868062 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7842ab7e-02d3-4944-b792-05b09bb407f7 service neutron] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:20:16.878058 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7842ab7e-02d3-4944-b792-05b09bb407f7 service neutron] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:20:16.937001 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-79a808ad-540c-4023-a573-6311af8eb740 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:20:16.937882 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a808ad-540c-4023-a573-6311af8eb740 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:20:16.938202 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a808ad-540c-4023-a573-6311af8eb740 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:20:16.938461 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-79a808ad-540c-4023-a573-6311af8eb740 admin admin] PATH_INFO: `/v3/users/3c027f34670e4b9ca3568baed7786296` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:20:16.939834 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a808ad-540c-4023-a573-6311af8eb740 admin admin] RBAC: Authorizing `identity:delete_user(user_id=3c027f34670e4b9ca3568baed7786296)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:20:16.941769 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-79a808ad-540c-4023-a573-6311af8eb740 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:20:16.995560 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-79a808ad-540c-4023-a573-6311af8eb740 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '3c027f34670e4b9ca3568baed7786296'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:20:17.004093 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-79a808ad-540c-4023-a573-6311af8eb740 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '3c027f34670e4b9ca3568baed7786296'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:20:17.016398 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-79a808ad-540c-4023-a573-6311af8eb740 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '3c027f34670e4b9ca3568baed7786296'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:20:17.016968 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-79a808ad-540c-4023-a573-6311af8eb740 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '3c027f34670e4b9ca3568baed7786296'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:20:17.017591 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-79a808ad-540c-4023-a573-6311af8eb740 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '3c027f34670e4b9ca3568baed7786296'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:20:17.029702 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1174/2353] 10.222.0.52 () {58 vars in 1223 bytes} [Wed Jul 13 22:20:16 2022] DELETE /identity/v3/users/3c027f34670e4b9ca3568baed7786296 => generated 0 bytes in 162 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 22:20:17.358302 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4cd0f9-9794-43d0-8745-3cd3251874c0 admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:20:17.359807 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-aa4cd0f9-9794-43d0-8745-3cd3251874c0 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:20:17.373480 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-aa4cd0f9-9794-43d0-8745-3cd3251874c0 admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:20:17.436506 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e3de5a79-9406-488c-9ceb-6cf0e71e5813 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:20:17.437388 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3de5a79-9406-488c-9ceb-6cf0e71e5813 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:20:17.437670 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3de5a79-9406-488c-9ceb-6cf0e71e5813 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:20:17.438021 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e3de5a79-9406-488c-9ceb-6cf0e71e5813 admin admin] PATH_INFO: `/v3/projects/34078dc8acc54dcfa1ae5f3a0c5fbf20` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:20:17.439285 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3de5a79-9406-488c-9ceb-6cf0e71e5813 admin admin] RBAC: Authorizing `identity:delete_project(project_id=34078dc8acc54dcfa1ae5f3a0c5fbf20)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:20:17.441248 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e3de5a79-9406-488c-9ceb-6cf0e71e5813 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:20:17.491019 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-e3de5a79-9406-488c-9ceb-6cf0e71e5813 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '34078dc8acc54dcfa1ae5f3a0c5fbf20', 'request_id': 'req-e3de5a79-9406-488c-9ceb-6cf0e71e5813'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:20:17.497581 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-e3de5a79-9406-488c-9ceb-6cf0e71e5813 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '34078dc8acc54dcfa1ae5f3a0c5fbf20', 'request_id': 'req-e3de5a79-9406-488c-9ceb-6cf0e71e5813'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:20:17.499680 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=87,keystone:DELETE=12,keystone:INSERT=1 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:20:17.504678 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-e3de5a79-9406-488c-9ceb-6cf0e71e5813 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '34078dc8acc54dcfa1ae5f3a0c5fbf20', 'request_id': 'req-e3de5a79-9406-488c-9ceb-6cf0e71e5813'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:20:17.519698 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-e3de5a79-9406-488c-9ceb-6cf0e71e5813 admin admin] The token cache is being invalidate because project 34078dc8acc54dcfa1ae5f3a0c5fbf20 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80381) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 22:20:17.519989 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-e3de5a79-9406-488c-9ceb-6cf0e71e5813 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:20:17.520617 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-e3de5a79-9406-488c-9ceb-6cf0e71e5813 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:20:17.522392 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1180/2354] 10.222.0.52 () {58 vars in 1232 bytes} [Wed Jul 13 22:20:17 2022] DELETE /identity/v3/projects/34078dc8acc54dcfa1ae5f3a0c5fbf20 => generated 0 bytes in 164 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 22:20:17.527285 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db83f9e9-2438-4688-ae0f-71ebcd7f7608 None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:20:17.527811 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db83f9e9-2438-4688-ae0f-71ebcd7f7608 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:20:17.527811 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-db83f9e9-2438-4688-ae0f-71ebcd7f7608 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:20:17.549719 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-db83f9e9-2438-4688-ae0f-71ebcd7f7608 None None] MFA Rules not processed for user `cc99c93da5074cf5a48bc2aa9a92c5e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 22:20:17.550981 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-db83f9e9-2438-4688-ae0f-71ebcd7f7608 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:20:17.555393 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:20:17.617798 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1175/2355] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jul 13 22:20:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 22:20:17.626090 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1f0038-3919-4320-845d-fa7a97163bc2 None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:20:17.626621 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1f0038-3919-4320-845d-fa7a97163bc2 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:20:17.626954 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4f1f0038-3919-4320-845d-fa7a97163bc2 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:20:17.649262 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-4f1f0038-3919-4320-845d-fa7a97163bc2 None None] MFA Rules not processed for user `cc99c93da5074cf5a48bc2aa9a92c5e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 22:20:17.650477 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-4f1f0038-3919-4320-845d-fa7a97163bc2 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:20:17.659099 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1181/2356] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jul 13 22:20:17 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 22:20:17.663419 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db83f9e9-2438-4688-ae0f-71ebcd7f7608 None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:20:17.674766 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-db83f9e9-2438-4688-ae0f-71ebcd7f7608 None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:20:17.679680 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffdd230-8148-41d1-87f3-962a9b8238c1 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:20:17.680356 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffdd230-8148-41d1-87f3-962a9b8238c1 admin admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:20:17.680639 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffdd230-8148-41d1-87f3-962a9b8238c1 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:20:17.680915 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-bffdd230-8148-41d1-87f3-962a9b8238c1 admin admin] PATH_INFO: `/v3/domains` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:20:17.681663 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bffdd230-8148-41d1-87f3-962a9b8238c1 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 22:20:17.682336 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bffdd230-8148-41d1-87f3-962a9b8238c1 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:20:17.683630 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-bffdd230-8148-41d1-87f3-962a9b8238c1 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:20:17.697144 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1176/2357] 10.222.0.52 () {58 vars in 1152 bytes} [Wed Jul 13 22:20:17 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 34 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 22:20:17.701670 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1f0038-3919-4320-845d-fa7a97163bc2 None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:20:17.712221 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4f1f0038-3919-4320-845d-fa7a97163bc2 None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:20:17.716911 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d237d1ae-b2ae-436b-ab40-6421c5f24d46 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:20:17.717577 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d237d1ae-b2ae-436b-ab40-6421c5f24d46 admin admin] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:20:17.717922 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d237d1ae-b2ae-436b-ab40-6421c5f24d46 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:20:17.718121 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d237d1ae-b2ae-436b-ab40-6421c5f24d46 admin admin] PATH_INFO: `/v3/projects` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:20:17.718862 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d237d1ae-b2ae-436b-ab40-6421c5f24d46 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:20:17.721154 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d237d1ae-b2ae-436b-ab40-6421c5f24d46 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:20:17.749576 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1182/2358] 10.222.0.52 () {60 vars in 1152 bytes} [Wed Jul 13 22:20:17 2022] POST /identity/v3/projects => generated 387 bytes in 48 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 22:20:17.754150 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffdd230-8148-41d1-87f3-962a9b8238c1 admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:20:17.764655 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-bffdd230-8148-41d1-87f3-962a9b8238c1 admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:20:17.826645 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9cf566e-28b7-48c3-9e83-4510b78b39dd admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:20:17.827517 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cf566e-28b7-48c3-9e83-4510b78b39dd admin admin] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:20:17.827908 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cf566e-28b7-48c3-9e83-4510b78b39dd admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:20:17.828196 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-b9cf566e-28b7-48c3-9e83-4510b78b39dd admin admin] PATH_INFO: `/v3/users` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:20:17.828951 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9cf566e-28b7-48c3-9e83-4510b78b39dd admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:20:17.830923 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-b9cf566e-28b7-48c3-9e83-4510b78b39dd admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:20:17.859546 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1177/2359] 10.222.0.52 () {60 vars in 1143 bytes} [Wed Jul 13 22:20:17 2022] POST /identity/v3/users => generated 294 bytes in 106 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 22:20:17.863636 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d237d1ae-b2ae-436b-ab40-6421c5f24d46 admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:20:17.874287 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d237d1ae-b2ae-436b-ab40-6421c5f24d46 admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:20:17.879207 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef1987d-322a-4304-b1b6-afacec25c4a6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:20:17.879878 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef1987d-322a-4304-b1b6-afacec25c4a6 admin admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:20:17.880161 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef1987d-322a-4304-b1b6-afacec25c4a6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:20:17.880431 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8ef1987d-322a-4304-b1b6-afacec25c4a6 admin admin] PATH_INFO: `/v3/roles` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:20:17.881168 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef1987d-322a-4304-b1b6-afacec25c4a6 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:20:17.882820 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8ef1987d-322a-4304-b1b6-afacec25c4a6 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:20:17.894577 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1183/2360] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jul 13 22:20:17 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 22:20:17.898824 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9cf566e-28b7-48c3-9e83-4510b78b39dd admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:20:17.908931 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-b9cf566e-28b7-48c3-9e83-4510b78b39dd admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:20:17.915158 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-becb55ea-3590-416d-88cf-e92e83892014 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:20:17.915820 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becb55ea-3590-416d-88cf-e92e83892014 admin admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:20:17.916096 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becb55ea-3590-416d-88cf-e92e83892014 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:20:17.916368 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-becb55ea-3590-416d-88cf-e92e83892014 admin admin] PATH_INFO: `/v3/roles` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:20:17.917090 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-becb55ea-3590-416d-88cf-e92e83892014 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:20:17.918711 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-becb55ea-3590-416d-88cf-e92e83892014 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:20:17.929962 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1178/2361] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jul 13 22:20:17 2022] GET /identity/v3/roles => generated 1420 bytes in 32 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 22:20:17.934133 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef1987d-322a-4304-b1b6-afacec25c4a6 admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:20:17.944515 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8ef1987d-322a-4304-b1b6-afacec25c4a6 admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:20:17.949511 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f16fa97-f313-4a5b-a229-8a707141f2b4 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:20:17.950409 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f16fa97-f313-4a5b-a229-8a707141f2b4 admin admin] REQUEST_METHOD: `PUT` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:20:17.950796 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f16fa97-f313-4a5b-a229-8a707141f2b4 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:20:17.951089 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7f16fa97-f313-4a5b-a229-8a707141f2b4 admin admin] PATH_INFO: `/v3/projects/25400ee014ff4c2a853e82c313f9828f/users/5a066eaa147448caa2b0a8a549df813e/roles/2a5e86976394498abb6310697cdb3f8b` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:20:17.966095 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f16fa97-f313-4a5b-a229-8a707141f2b4 admin admin] RBAC: Authorizing `identity:create_grant(project_id=25400ee014ff4c2a853e82c313f9828f, user_id=5a066eaa147448caa2b0a8a549df813e, role_id=2a5e86976394498abb6310697cdb3f8b)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:20:17.968803 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7f16fa97-f313-4a5b-a229-8a707141f2b4 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:20:17.983838 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1184/2362] 10.222.0.52 () {60 vars in 1482 bytes} [Wed Jul 13 22:20:17 2022] PUT /identity/v3/projects/25400ee014ff4c2a853e82c313f9828f/users/5a066eaa147448caa2b0a8a549df813e/roles/2a5e86976394498abb6310697cdb3f8b => generated 0 bytes in 50 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 22:20:18.005370 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9958373-087e-4b13-acf8-9ad21568ea0e None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:20:18.005750 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9958373-087e-4b13-acf8-9ad21568ea0e None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:20:18.006107 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d9958373-087e-4b13-acf8-9ad21568ea0e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:20:18.036973 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-d9958373-087e-4b13-acf8-9ad21568ea0e None None] MFA Rules not processed for user `5a066eaa147448caa2b0a8a549df813e`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 22:20:18.038198 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-d9958373-087e-4b13-acf8-9ad21568ea0e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:20:18.119283 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1179/2363] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jul 13 22:20:17 2022] POST /identity/v3/auth/tokens => generated 3254 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 22:20:18.126455 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f16fa97-f313-4a5b-a229-8a707141f2b4 admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:20:18.199063 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7f16fa97-f313-4a5b-a229-8a707141f2b4 admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:20:18.203850 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbdefcbd-b6d3-4ca6-b1d7-0ff1464972d0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:20:18.204514 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdefcbd-b6d3-4ca6-b1d7-0ff1464972d0 admin admin] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:20:18.204789 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdefcbd-b6d3-4ca6-b1d7-0ff1464972d0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:20:18.205060 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dbdefcbd-b6d3-4ca6-b1d7-0ff1464972d0 admin admin] PATH_INFO: `/v3/projects` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:20:18.205744 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbdefcbd-b6d3-4ca6-b1d7-0ff1464972d0 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:20:18.207685 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dbdefcbd-b6d3-4ca6-b1d7-0ff1464972d0 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:20:18.238086 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1185/2364] 10.222.0.52 () {60 vars in 1152 bytes} [Wed Jul 13 22:20:18 2022] POST /identity/v3/projects => generated 387 bytes in 111 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 22:20:18.240403 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9958373-087e-4b13-acf8-9ad21568ea0e None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:20:18.247104 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=97,keystone:INSERT=3 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:20:18.251901 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d9958373-087e-4b13-acf8-9ad21568ea0e None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:20:18.312839 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9033c9-3239-4f51-9d45-2046eb5daa90 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:20:18.313608 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9033c9-3239-4f51-9d45-2046eb5daa90 admin admin] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:20:18.313888 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9033c9-3239-4f51-9d45-2046eb5daa90 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:20:18.314155 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2e9033c9-3239-4f51-9d45-2046eb5daa90 admin admin] PATH_INFO: `/v3/users` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:20:18.314900 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e9033c9-3239-4f51-9d45-2046eb5daa90 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:20:18.316825 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2e9033c9-3239-4f51-9d45-2046eb5daa90 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:20:18.339487 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1180/2365] 10.222.0.52 () {60 vars in 1143 bytes} [Wed Jul 13 22:20:18 2022] POST /identity/v3/users => generated 300 bytes in 100 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 22:20:18.343605 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbdefcbd-b6d3-4ca6-b1d7-0ff1464972d0 admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:20:18.354476 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dbdefcbd-b6d3-4ca6-b1d7-0ff1464972d0 admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:20:18.362444 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b68e35f-9e8b-486c-95cb-6c49faafc88f admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:20:18.363600 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b68e35f-9e8b-486c-95cb-6c49faafc88f admin admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:20:18.364095 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b68e35f-9e8b-486c-95cb-6c49faafc88f admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:20:18.364525 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b68e35f-9e8b-486c-95cb-6c49faafc88f admin admin] PATH_INFO: `/v3/roles` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:20:18.365665 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b68e35f-9e8b-486c-95cb-6c49faafc88f admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:20:18.367396 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b68e35f-9e8b-486c-95cb-6c49faafc88f admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:20:18.379241 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1186/2366] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jul 13 22:20:18 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 22:20:18.384266 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9033c9-3239-4f51-9d45-2046eb5daa90 admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:20:18.397752 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2e9033c9-3239-4f51-9d45-2046eb5daa90 admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:20:18.402576 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4fe978-8c65-43c0-a495-abf5a2099519 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:20:18.403463 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4fe978-8c65-43c0-a495-abf5a2099519 admin admin] REQUEST_METHOD: `PUT` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:20:18.403776 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4fe978-8c65-43c0-a495-abf5a2099519 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:20:18.404042 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b4fe978-8c65-43c0-a495-abf5a2099519 admin admin] PATH_INFO: `/v3/projects/ac43d5080bb44ee29fa172f580133514/users/68e294a7b407423aba919458e657df87/roles/44efba2579f342df8a0789c59d163ec3` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:20:18.418745 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b4fe978-8c65-43c0-a495-abf5a2099519 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ac43d5080bb44ee29fa172f580133514, user_id=68e294a7b407423aba919458e657df87, role_id=44efba2579f342df8a0789c59d163ec3)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:20:18.421394 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5b4fe978-8c65-43c0-a495-abf5a2099519 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:20:18.435734 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1181/2367] 10.222.0.52 () {60 vars in 1482 bytes} [Wed Jul 13 22:20:18 2022] PUT /identity/v3/projects/ac43d5080bb44ee29fa172f580133514/users/68e294a7b407423aba919458e657df87/roles/44efba2579f342df8a0789c59d163ec3 => generated 0 bytes in 53 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 22:20:18.440054 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b68e35f-9e8b-486c-95cb-6c49faafc88f admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:20:18.450924 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b68e35f-9e8b-486c-95cb-6c49faafc88f admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:20:18.505821 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=94,keystone:DELETE=2,keystone:INSERT=4 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:20:18.515184 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a0e3bab0-0035-4865-b282-4a84cf116d34 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:20:18.515887 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e3bab0-0035-4865-b282-4a84cf116d34 admin admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:20:18.516217 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e3bab0-0035-4865-b282-4a84cf116d34 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:20:18.517020 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a0e3bab0-0035-4865-b282-4a84cf116d34 admin admin] PATH_INFO: `/v3/roles` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:20:18.517788 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e3bab0-0035-4865-b282-4a84cf116d34 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:20:18.519766 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a0e3bab0-0035-4865-b282-4a84cf116d34 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:20:18.529714 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1187/2368] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jul 13 22:20:18 2022] GET /identity/v3/roles => generated 1420 bytes in 90 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 22:20:18.534203 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4fe978-8c65-43c0-a495-abf5a2099519 admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:20:18.544721 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5b4fe978-8c65-43c0-a495-abf5a2099519 admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:20:18.550882 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d85d3b4-a814-4382-bd46-918618fd6ed8 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:20:18.551820 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d85d3b4-a814-4382-bd46-918618fd6ed8 admin admin] REQUEST_METHOD: `PUT` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:20:18.552098 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d85d3b4-a814-4382-bd46-918618fd6ed8 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:20:18.552370 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2d85d3b4-a814-4382-bd46-918618fd6ed8 admin admin] PATH_INFO: `/v3/projects/ac43d5080bb44ee29fa172f580133514/users/68e294a7b407423aba919458e657df87/roles/44efba2579f342df8a0789c59d163ec3` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:20:18.554695 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d85d3b4-a814-4382-bd46-918618fd6ed8 admin admin] RBAC: Authorizing `identity:create_grant(project_id=ac43d5080bb44ee29fa172f580133514, user_id=68e294a7b407423aba919458e657df87, role_id=44efba2579f342df8a0789c59d163ec3)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:20:18.557483 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2d85d3b4-a814-4382-bd46-918618fd6ed8 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:20:18.569066 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1182/2369] 10.222.0.52 () {60 vars in 1482 bytes} [Wed Jul 13 22:20:18 2022] PUT /identity/v3/projects/ac43d5080bb44ee29fa172f580133514/users/68e294a7b407423aba919458e657df87/roles/44efba2579f342df8a0789c59d163ec3 => generated 0 bytes in 35 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 22:20:18.580798 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebca92df-67a4-463d-8b9b-0798caeaf1d3 None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:20:18.581171 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebca92df-67a4-463d-8b9b-0798caeaf1d3 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:20:18.581636 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ebca92df-67a4-463d-8b9b-0798caeaf1d3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:20:18.617709 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-ebca92df-67a4-463d-8b9b-0798caeaf1d3 None None] MFA Rules not processed for user `68e294a7b407423aba919458e657df87`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 22:20:18.618958 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-ebca92df-67a4-463d-8b9b-0798caeaf1d3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:20:18.690163 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1188/2370] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jul 13 22:20:18 2022] POST /identity/v3/auth/tokens => generated 3321 bytes in 111 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 22:20:18.705587 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d85d3b4-a814-4382-bd46-918618fd6ed8 admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:20:18.707692 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-2d85d3b4-a814-4382-bd46-918618fd6ed8 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:20:18.768731 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2d85d3b4-a814-4382-bd46-918618fd6ed8 admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:20:18.773165 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07728e33-c5e2-4e73-bef3-5d3502a173a6 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc77e1ea1ada4700b809beb1c980c5e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417e3e2f1333481795a089bef2e24bea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:20:18.773811 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07728e33-c5e2-4e73-bef3-5d3502a173a6 service cinder] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:20:18.774080 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07728e33-c5e2-4e73-bef3-5d3502a173a6 service cinder] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:20:18.774347 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-07728e33-c5e2-4e73-bef3-5d3502a173a6 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:20:18.784034 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07728e33-c5e2-4e73-bef3-5d3502a173a6 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:20:18.785469 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-07728e33-c5e2-4e73-bef3-5d3502a173a6 service cinder] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:20:18.795073 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1183/2371] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jul 13 22:20:18 2022] GET /identity/v3/auth/tokens => generated 3321 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 22:20:19.373641 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebca92df-67a4-463d-8b9b-0798caeaf1d3 None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:20:19.444224 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ebca92df-67a4-463d-8b9b-0798caeaf1d3 None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:20:19.451617 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8191c2fc-8027-4ecc-901b-04d6151752d2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:20:19.452652 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8191c2fc-8027-4ecc-901b-04d6151752d2 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:20:19.453043 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8191c2fc-8027-4ecc-901b-04d6151752d2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:20:19.453432 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8191c2fc-8027-4ecc-901b-04d6151752d2 admin admin] PATH_INFO: `/v3/users/5a066eaa147448caa2b0a8a549df813e` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:20:19.454789 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8191c2fc-8027-4ecc-901b-04d6151752d2 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5a066eaa147448caa2b0a8a549df813e)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:20:19.456713 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8191c2fc-8027-4ecc-901b-04d6151752d2 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:20:19.519411 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-8191c2fc-8027-4ecc-901b-04d6151752d2 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5a066eaa147448caa2b0a8a549df813e'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:20:19.529452 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-8191c2fc-8027-4ecc-901b-04d6151752d2 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5a066eaa147448caa2b0a8a549df813e'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:20:19.551134 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-8191c2fc-8027-4ecc-901b-04d6151752d2 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5a066eaa147448caa2b0a8a549df813e'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:20:19.552393 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-8191c2fc-8027-4ecc-901b-04d6151752d2 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5a066eaa147448caa2b0a8a549df813e'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:20:19.553140 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-8191c2fc-8027-4ecc-901b-04d6151752d2 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5a066eaa147448caa2b0a8a549df813e'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:20:19.554328 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07728e33-c5e2-4e73-bef3-5d3502a173a6 service cinder] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:20:19.557026 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-07728e33-c5e2-4e73-bef3-5d3502a173a6 service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:20:19.561145 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:20:19.574458 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1189/2372] 10.222.0.52 () {58 vars in 1223 bytes} [Wed Jul 13 22:20:19 2022] DELETE /identity/v3/users/5a066eaa147448caa2b0a8a549df813e => generated 0 bytes in 200 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 22:20:19.578272 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd1f1d9-ce51-4714-afcb-f0fbb129ae7a None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:20:19.578718 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd1f1d9-ce51-4714-afcb-f0fbb129ae7a None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:20:19.579235 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7cd1f1d9-ce51-4714-afcb-f0fbb129ae7a None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:20:19.598774 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-7cd1f1d9-ce51-4714-afcb-f0fbb129ae7a None None] MFA Rules not processed for user `cc99c93da5074cf5a48bc2aa9a92c5e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 22:20:19.600070 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-7cd1f1d9-ce51-4714-afcb-f0fbb129ae7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:20:19.628473 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:20:19.658766 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1190/2373] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jul 13 22:20:19 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 82 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 13 22:20:19.658977 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-07728e33-c5e2-4e73-bef3-5d3502a173a6 service cinder] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:20:19.668635 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd1f1d9-ce51-4714-afcb-f0fbb129ae7a None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:20:19.670164 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-7cd1f1d9-ce51-4714-afcb-f0fbb129ae7a None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:20:19.721887 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9b99c9-58d4-4a60-a53e-e3fdf0b50bb3 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac673d641d8849c79bf4cf71c9805587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417e3e2f1333481795a089bef2e24bea', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:20:19.722599 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9b99c9-58d4-4a60-a53e-e3fdf0b50bb3 service placement] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:20:19.722886 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9b99c9-58d4-4a60-a53e-e3fdf0b50bb3 service placement] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:20:19.723163 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ce9b99c9-58d4-4a60-a53e-e3fdf0b50bb3 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:20:19.725067 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-ce9b99c9-58d4-4a60-a53e-e3fdf0b50bb3 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:20:19.731007 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7cd1f1d9-ce51-4714-afcb-f0fbb129ae7a None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:20:19.734777 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80659c6e-9aa5-4a6e-9175-40c90d9ad642 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '847ec9a1bd4d4dec804318a5c845895a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417e3e2f1333481795a089bef2e24bea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:20:19.735568 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80659c6e-9aa5-4a6e-9175-40c90d9ad642 service neutron] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:20:19.735868 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80659c6e-9aa5-4a6e-9175-40c90d9ad642 service neutron] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:20:19.736132 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-80659c6e-9aa5-4a6e-9175-40c90d9ad642 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:20:19.736861 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9b99c9-58d4-4a60-a53e-e3fdf0b50bb3 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:20:19.738512 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ce9b99c9-58d4-4a60-a53e-e3fdf0b50bb3 service placement] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:20:19.746121 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80659c6e-9aa5-4a6e-9175-40c90d9ad642 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:20:19.747581 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-80659c6e-9aa5-4a6e-9175-40c90d9ad642 service neutron] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:20:19.748767 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1184/2374] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jul 13 22:20:19 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 195 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 13 22:20:19.759481 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1191/2375] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jul 13 22:20:19 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 22:20:20.036360 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9b99c9-58d4-4a60-a53e-e3fdf0b50bb3 service placement] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:20:20.039888 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-ce9b99c9-58d4-4a60-a53e-e3fdf0b50bb3 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:20:20.054463 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ce9b99c9-58d4-4a60-a53e-e3fdf0b50bb3 service placement] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:20:20.059730 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6feb26f-356d-46f4-8df7-f2a44afd1a1b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:20:20.060693 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6feb26f-356d-46f4-8df7-f2a44afd1a1b admin admin] REQUEST_METHOD: `DELETE` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:20:20.060969 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6feb26f-356d-46f4-8df7-f2a44afd1a1b admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:20:20.061243 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6feb26f-356d-46f4-8df7-f2a44afd1a1b admin admin] PATH_INFO: `/v3/projects/25400ee014ff4c2a853e82c313f9828f` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:20:20.062462 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6feb26f-356d-46f4-8df7-f2a44afd1a1b admin admin] RBAC: Authorizing `identity:delete_project(project_id=25400ee014ff4c2a853e82c313f9828f)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:20:20.064575 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6feb26f-356d-46f4-8df7-f2a44afd1a1b admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:20:20.119599 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-d6feb26f-356d-46f4-8df7-f2a44afd1a1b admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '25400ee014ff4c2a853e82c313f9828f', 'request_id': 'req-d6feb26f-356d-46f4-8df7-f2a44afd1a1b'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:20:20.126129 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-d6feb26f-356d-46f4-8df7-f2a44afd1a1b admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '25400ee014ff4c2a853e82c313f9828f', 'request_id': 'req-d6feb26f-356d-46f4-8df7-f2a44afd1a1b'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:20:20.132243 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-d6feb26f-356d-46f4-8df7-f2a44afd1a1b admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '25400ee014ff4c2a853e82c313f9828f', 'request_id': 'req-d6feb26f-356d-46f4-8df7-f2a44afd1a1b'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:20:20.147336 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-d6feb26f-356d-46f4-8df7-f2a44afd1a1b admin admin] The token cache is being invalidate because project 25400ee014ff4c2a853e82c313f9828f was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80382) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 22:20:20.147651 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-d6feb26f-356d-46f4-8df7-f2a44afd1a1b admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:20:20.148312 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-d6feb26f-356d-46f4-8df7-f2a44afd1a1b admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:20:20.149963 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1185/2376] 10.222.0.52 () {58 vars in 1232 bytes} [Wed Jul 13 22:20:20 2022] DELETE /identity/v3/projects/25400ee014ff4c2a853e82c313f9828f => generated 0 bytes in 114 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 22:20:20.153937 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80659c6e-9aa5-4a6e-9175-40c90d9ad642 service neutron] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:20:20.164221 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-80659c6e-9aa5-4a6e-9175-40c90d9ad642 service neutron] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:20:20.232911 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6b55657-ea31-4e53-8912-9d76ea3b759c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:20:20.233889 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b55657-ea31-4e53-8912-9d76ea3b759c admin admin] REQUEST_METHOD: `DELETE` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:20:20.234195 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b55657-ea31-4e53-8912-9d76ea3b759c admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:20:20.234462 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d6b55657-ea31-4e53-8912-9d76ea3b759c admin admin] PATH_INFO: `/v3/users/68e294a7b407423aba919458e657df87` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:20:20.236043 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b55657-ea31-4e53-8912-9d76ea3b759c admin admin] RBAC: Authorizing `identity:delete_user(user_id=68e294a7b407423aba919458e657df87)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:20:20.238037 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-d6b55657-ea31-4e53-8912-9d76ea3b759c admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:20:20.283152 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=94,keystone:DELETE=6 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:20:20.290335 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-d6b55657-ea31-4e53-8912-9d76ea3b759c admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '68e294a7b407423aba919458e657df87'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:20:20.300919 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-d6b55657-ea31-4e53-8912-9d76ea3b759c admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '68e294a7b407423aba919458e657df87'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:20:20.316091 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-d6b55657-ea31-4e53-8912-9d76ea3b759c admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '68e294a7b407423aba919458e657df87'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:20:20.316792 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-d6b55657-ea31-4e53-8912-9d76ea3b759c admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '68e294a7b407423aba919458e657df87'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:20:20.317407 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-d6b55657-ea31-4e53-8912-9d76ea3b759c admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '68e294a7b407423aba919458e657df87'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:20:20.330984 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1192/2377] 10.222.0.52 () {58 vars in 1223 bytes} [Wed Jul 13 22:20:20 2022] DELETE /identity/v3/users/68e294a7b407423aba919458e657df87 => generated 0 bytes in 178 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 22:20:20.619702 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6feb26f-356d-46f4-8df7-f2a44afd1a1b admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:20:20.621355 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-d6feb26f-356d-46f4-8df7-f2a44afd1a1b admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:20:20.641618 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d6feb26f-356d-46f4-8df7-f2a44afd1a1b admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:20:20.693612 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=92,keystone:DELETE=7,keystone:INSERT=1 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:20:20.701804 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ff6b5aa7-de20-43bf-afe7-0e65fcda8b25 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:20:20.702712 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6b5aa7-de20-43bf-afe7-0e65fcda8b25 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:20:20.702998 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6b5aa7-de20-43bf-afe7-0e65fcda8b25 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:20:20.703269 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ff6b5aa7-de20-43bf-afe7-0e65fcda8b25 admin admin] PATH_INFO: `/v3/projects/ac43d5080bb44ee29fa172f580133514` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:20:20.705161 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6b5aa7-de20-43bf-afe7-0e65fcda8b25 admin admin] RBAC: Authorizing `identity:delete_project(project_id=ac43d5080bb44ee29fa172f580133514)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:20:20.707448 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-ff6b5aa7-de20-43bf-afe7-0e65fcda8b25 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:20:20.779966 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-ff6b5aa7-de20-43bf-afe7-0e65fcda8b25 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': 'ac43d5080bb44ee29fa172f580133514', 'request_id': 'req-ff6b5aa7-de20-43bf-afe7-0e65fcda8b25'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:20:20.786438 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-ff6b5aa7-de20-43bf-afe7-0e65fcda8b25 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': 'ac43d5080bb44ee29fa172f580133514', 'request_id': 'req-ff6b5aa7-de20-43bf-afe7-0e65fcda8b25'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:20:20.792152 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-ff6b5aa7-de20-43bf-afe7-0e65fcda8b25 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': 'ac43d5080bb44ee29fa172f580133514', 'request_id': 'req-ff6b5aa7-de20-43bf-afe7-0e65fcda8b25'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:20:20.806522 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-ff6b5aa7-de20-43bf-afe7-0e65fcda8b25 admin admin] The token cache is being invalidate because project ac43d5080bb44ee29fa172f580133514 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80382) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 22:20:20.806745 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-ff6b5aa7-de20-43bf-afe7-0e65fcda8b25 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:20:20.807474 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-ff6b5aa7-de20-43bf-afe7-0e65fcda8b25 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:20:20.809340 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1186/2378] 10.222.0.52 () {58 vars in 1232 bytes} [Wed Jul 13 22:20:20 2022] DELETE /identity/v3/projects/ac43d5080bb44ee29fa172f580133514 => generated 0 bytes in 190 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 22:20:20.816315 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83450af8-db54-4e0e-8c3c-375891e41fde None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:20:20.817170 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83450af8-db54-4e0e-8c3c-375891e41fde None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:20:20.817738 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-83450af8-db54-4e0e-8c3c-375891e41fde None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:20:20.841094 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-83450af8-db54-4e0e-8c3c-375891e41fde None None] MFA Rules not processed for user `cc99c93da5074cf5a48bc2aa9a92c5e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 22:20:20.842296 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-83450af8-db54-4e0e-8c3c-375891e41fde None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:20:20.906268 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1193/2379] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jul 13 22:20:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 92 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 22:20:20.914851 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9f244a-173c-4320-bd07-e2cc84716b9e None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:20:20.915143 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9f244a-173c-4320-bd07-e2cc84716b9e None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:20:20.915406 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-9d9f244a-173c-4320-bd07-e2cc84716b9e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:20:20.936024 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-9d9f244a-173c-4320-bd07-e2cc84716b9e None None] MFA Rules not processed for user `cc99c93da5074cf5a48bc2aa9a92c5e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 22:20:20.937401 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-9d9f244a-173c-4320-bd07-e2cc84716b9e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:20:20.950806 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1187/2380] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jul 13 22:20:20 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 37 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 22:20:20.953248 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83450af8-db54-4e0e-8c3c-375891e41fde None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:20:20.965149 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-83450af8-db54-4e0e-8c3c-375891e41fde None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:20:20.982921 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5251e23a-d3ef-475f-8b90-84a4619fa196 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:20:20.982921 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5251e23a-d3ef-475f-8b90-84a4619fa196 admin admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:20:20.982921 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5251e23a-d3ef-475f-8b90-84a4619fa196 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:20:20.982921 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5251e23a-d3ef-475f-8b90-84a4619fa196 admin admin] PATH_INFO: `/v3/domains` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:20:20.983263 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5251e23a-d3ef-475f-8b90-84a4619fa196 admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80381) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 22:20:20.983263 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5251e23a-d3ef-475f-8b90-84a4619fa196 admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:20:20.983263 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5251e23a-d3ef-475f-8b90-84a4619fa196 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:20:21.000812 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1194/2381] 10.222.0.52 () {58 vars in 1152 bytes} [Wed Jul 13 22:20:20 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 48 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 22:20:21.005022 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9f244a-173c-4320-bd07-e2cc84716b9e None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:20:21.016747 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-9d9f244a-173c-4320-bd07-e2cc84716b9e None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:20:21.022611 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba2fe97-6a31-4f61-8e19-ba5317672779 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:20:21.023329 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba2fe97-6a31-4f61-8e19-ba5317672779 admin admin] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:20:21.023669 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba2fe97-6a31-4f61-8e19-ba5317672779 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:20:21.024006 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4ba2fe97-6a31-4f61-8e19-ba5317672779 admin admin] PATH_INFO: `/v3/projects` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:20:21.025455 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba2fe97-6a31-4f61-8e19-ba5317672779 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:20:21.035191 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4ba2fe97-6a31-4f61-8e19-ba5317672779 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:20:21.069603 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1188/2382] 10.222.0.52 () {60 vars in 1152 bytes} [Wed Jul 13 22:20:21 2022] POST /identity/v3/projects => generated 371 bytes in 65 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 22:20:21.074373 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5251e23a-d3ef-475f-8b90-84a4619fa196 admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:20:21.085244 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5251e23a-d3ef-475f-8b90-84a4619fa196 admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:20:21.144686 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7378153-bb7d-4de7-943e-031def69a61e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:20:21.145365 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7378153-bb7d-4de7-943e-031def69a61e admin admin] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:20:21.145645 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7378153-bb7d-4de7-943e-031def69a61e admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:20:21.145944 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f7378153-bb7d-4de7-943e-031def69a61e admin admin] PATH_INFO: `/v3/users` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:20:21.146757 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7378153-bb7d-4de7-943e-031def69a61e admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:20:21.148731 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f7378153-bb7d-4de7-943e-031def69a61e admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:20:21.172809 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1195/2383] 10.222.0.52 () {60 vars in 1143 bytes} [Wed Jul 13 22:20:21 2022] POST /identity/v3/users => generated 286 bytes in 99 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 22:20:21.177220 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba2fe97-6a31-4f61-8e19-ba5317672779 admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:20:21.190295 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4ba2fe97-6a31-4f61-8e19-ba5317672779 admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:20:21.196465 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f35ac62-e9b0-4159-b659-99d84be29a03 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:20:21.197354 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f35ac62-e9b0-4159-b659-99d84be29a03 admin admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:20:21.197721 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f35ac62-e9b0-4159-b659-99d84be29a03 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:20:21.198106 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f35ac62-e9b0-4159-b659-99d84be29a03 admin admin] PATH_INFO: `/v3/roles` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:20:21.198946 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f35ac62-e9b0-4159-b659-99d84be29a03 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:20:21.200695 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f35ac62-e9b0-4159-b659-99d84be29a03 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:20:21.212719 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1189/2384] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jul 13 22:20:21 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 22:20:21.217820 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7378153-bb7d-4de7-943e-031def69a61e admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:20:21.231809 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f7378153-bb7d-4de7-943e-031def69a61e admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:20:21.237570 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5042ef08-47fd-43a6-b04b-4f3a897f4314 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:20:21.238380 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5042ef08-47fd-43a6-b04b-4f3a897f4314 admin admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:20:21.238777 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5042ef08-47fd-43a6-b04b-4f3a897f4314 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:20:21.239142 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5042ef08-47fd-43a6-b04b-4f3a897f4314 admin admin] PATH_INFO: `/v3/roles` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:20:21.240005 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5042ef08-47fd-43a6-b04b-4f3a897f4314 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:20:21.241730 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5042ef08-47fd-43a6-b04b-4f3a897f4314 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:20:21.255049 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1196/2385] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jul 13 22:20:21 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 22:20:21.258998 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f35ac62-e9b0-4159-b659-99d84be29a03 admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:20:21.271827 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f35ac62-e9b0-4159-b659-99d84be29a03 admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:20:21.276854 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9f94bc-4288-4907-aa0c-83473ef51862 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:20:21.277818 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9f94bc-4288-4907-aa0c-83473ef51862 admin admin] REQUEST_METHOD: `PUT` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:20:21.278162 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9f94bc-4288-4907-aa0c-83473ef51862 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:20:21.278525 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1c9f94bc-4288-4907-aa0c-83473ef51862 admin admin] PATH_INFO: `/v3/projects/4b816123ed564cdca5b0b35454176fa1/users/8bf3c57f7c7b4ccfa71827e5f6b32acb/roles/2a5e86976394498abb6310697cdb3f8b` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:20:21.295187 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9f94bc-4288-4907-aa0c-83473ef51862 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4b816123ed564cdca5b0b35454176fa1, user_id=8bf3c57f7c7b4ccfa71827e5f6b32acb, role_id=2a5e86976394498abb6310697cdb3f8b)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:20:21.298157 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1c9f94bc-4288-4907-aa0c-83473ef51862 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:20:21.312058 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1190/2386] 10.222.0.52 () {60 vars in 1482 bytes} [Wed Jul 13 22:20:21 2022] PUT /identity/v3/projects/4b816123ed564cdca5b0b35454176fa1/users/8bf3c57f7c7b4ccfa71827e5f6b32acb/roles/2a5e86976394498abb6310697cdb3f8b => generated 0 bytes in 54 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 13 22:20:21.324530 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395d5678-6559-4b2b-b722-fd8be72ea4b6 None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:20:21.324821 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395d5678-6559-4b2b-b722-fd8be72ea4b6 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:20:21.325099 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-395d5678-6559-4b2b-b722-fd8be72ea4b6 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:20:21.355709 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-395d5678-6559-4b2b-b722-fd8be72ea4b6 None None] MFA Rules not processed for user `8bf3c57f7c7b4ccfa71827e5f6b32acb`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 22:20:21.357224 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-395d5678-6559-4b2b-b722-fd8be72ea4b6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:20:21.370837 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=92,keystone:DELETE=4,keystone:INSERT=4 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:20:21.430276 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1197/2387] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jul 13 22:20:21 2022] POST /identity/v3/auth/tokens => generated 3238 bytes in 107 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 13 22:20:21.434919 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9f94bc-4288-4907-aa0c-83473ef51862 admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:20:21.500440 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1c9f94bc-4288-4907-aa0c-83473ef51862 admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:20:21.506263 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53297d1-0e42-4067-bd2b-350e85f2f980 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:20:21.506955 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53297d1-0e42-4067-bd2b-350e85f2f980 admin admin] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:20:21.507232 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53297d1-0e42-4067-bd2b-350e85f2f980 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:20:21.507505 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f53297d1-0e42-4067-bd2b-350e85f2f980 admin admin] PATH_INFO: `/v3/projects` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:20:21.508199 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53297d1-0e42-4067-bd2b-350e85f2f980 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:20:21.510139 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f53297d1-0e42-4067-bd2b-350e85f2f980 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:20:21.539682 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1191/2388] 10.222.0.52 () {60 vars in 1152 bytes} [Wed Jul 13 22:20:21 2022] POST /identity/v3/projects => generated 373 bytes in 105 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 22:20:21.543939 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395d5678-6559-4b2b-b722-fd8be72ea4b6 None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:20:21.556027 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-395d5678-6559-4b2b-b722-fd8be72ea4b6 None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:20:21.613224 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af17f86-370f-4ee5-bb87-792658f83c0b admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:20:21.613901 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af17f86-370f-4ee5-bb87-792658f83c0b admin admin] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:20:21.614172 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af17f86-370f-4ee5-bb87-792658f83c0b admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:20:21.614443 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5af17f86-370f-4ee5-bb87-792658f83c0b admin admin] PATH_INFO: `/v3/users` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:20:21.615191 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af17f86-370f-4ee5-bb87-792658f83c0b admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:20:21.617129 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5af17f86-370f-4ee5-bb87-792658f83c0b admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:20:21.640187 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1198/2389] 10.222.0.52 () {60 vars in 1143 bytes} [Wed Jul 13 22:20:21 2022] POST /identity/v3/users => generated 293 bytes in 97 msecs (HTTP/1.1 201) 5 headers in 182 bytes (2 switches on core 0) Jul 13 22:20:21.644962 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53297d1-0e42-4067-bd2b-350e85f2f980 admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:20:21.661502 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f53297d1-0e42-4067-bd2b-350e85f2f980 admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:20:21.667465 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb7d31d-e38d-4fca-822f-b21e9ccd1aa0 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:20:21.668124 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb7d31d-e38d-4fca-822f-b21e9ccd1aa0 admin admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:20:21.668403 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb7d31d-e38d-4fca-822f-b21e9ccd1aa0 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:20:21.668672 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-beb7d31d-e38d-4fca-822f-b21e9ccd1aa0 admin admin] PATH_INFO: `/v3/roles` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:20:21.669413 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb7d31d-e38d-4fca-822f-b21e9ccd1aa0 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:20:21.671055 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-beb7d31d-e38d-4fca-822f-b21e9ccd1aa0 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:20:21.672382 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=89,keystone:DELETE=7,keystone:INSERT=4 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:20:21.688023 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1192/2390] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jul 13 22:20:21 2022] GET /identity/v3/roles => generated 1420 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 22:20:21.692467 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af17f86-370f-4ee5-bb87-792658f83c0b admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:20:21.703200 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5af17f86-370f-4ee5-bb87-792658f83c0b admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:20:21.708303 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db107e9-f2fa-4d8b-871a-1746e411a7f6 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:20:21.709199 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db107e9-f2fa-4d8b-871a-1746e411a7f6 admin admin] REQUEST_METHOD: `PUT` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:20:21.709486 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db107e9-f2fa-4d8b-871a-1746e411a7f6 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:20:21.709765 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1db107e9-f2fa-4d8b-871a-1746e411a7f6 admin admin] PATH_INFO: `/v3/projects/4f783cd7b91c459787f75abc3f0ccc5c/users/5eaa6a590eb14e8c80fde3fe19cbb8c4/roles/44efba2579f342df8a0789c59d163ec3` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:20:21.726012 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db107e9-f2fa-4d8b-871a-1746e411a7f6 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4f783cd7b91c459787f75abc3f0ccc5c, user_id=5eaa6a590eb14e8c80fde3fe19cbb8c4, role_id=44efba2579f342df8a0789c59d163ec3)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:20:21.728814 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1db107e9-f2fa-4d8b-871a-1746e411a7f6 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:20:21.743203 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1199/2391] 10.222.0.52 () {60 vars in 1482 bytes} [Wed Jul 13 22:20:21 2022] PUT /identity/v3/projects/4f783cd7b91c459787f75abc3f0ccc5c/users/5eaa6a590eb14e8c80fde3fe19cbb8c4/roles/44efba2579f342df8a0789c59d163ec3 => generated 0 bytes in 51 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 22:20:21.748511 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb7d31d-e38d-4fca-822f-b21e9ccd1aa0 admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:20:21.759532 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-beb7d31d-e38d-4fca-822f-b21e9ccd1aa0 admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:20:21.822269 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-2b12cf31-dec5-45a5-8817-506d8d0264eb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:20:21.822898 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b12cf31-dec5-45a5-8817-506d8d0264eb admin admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:20:21.823174 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b12cf31-dec5-45a5-8817-506d8d0264eb admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:20:21.823447 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2b12cf31-dec5-45a5-8817-506d8d0264eb admin admin] PATH_INFO: `/v3/roles` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:20:21.824169 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b12cf31-dec5-45a5-8817-506d8d0264eb admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:20:21.825764 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-2b12cf31-dec5-45a5-8817-506d8d0264eb admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:20:21.835638 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1193/2392] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jul 13 22:20:21 2022] GET /identity/v3/roles => generated 1420 bytes in 88 msecs (HTTP/1.1 200) 5 headers in 178 bytes (2 switches on core 0) Jul 13 22:20:21.840220 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db107e9-f2fa-4d8b-871a-1746e411a7f6 admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:20:21.850991 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1db107e9-f2fa-4d8b-871a-1746e411a7f6 admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:20:21.856371 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82d2578-bdc2-4bef-8530-ff207f593ba9 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:20:21.857288 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82d2578-bdc2-4bef-8530-ff207f593ba9 admin admin] REQUEST_METHOD: `PUT` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:20:21.857577 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82d2578-bdc2-4bef-8530-ff207f593ba9 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:20:21.857846 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c82d2578-bdc2-4bef-8530-ff207f593ba9 admin admin] PATH_INFO: `/v3/projects/4f783cd7b91c459787f75abc3f0ccc5c/users/5eaa6a590eb14e8c80fde3fe19cbb8c4/roles/44efba2579f342df8a0789c59d163ec3` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:20:21.859980 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82d2578-bdc2-4bef-8530-ff207f593ba9 admin admin] RBAC: Authorizing `identity:create_grant(project_id=4f783cd7b91c459787f75abc3f0ccc5c, user_id=5eaa6a590eb14e8c80fde3fe19cbb8c4, role_id=44efba2579f342df8a0789c59d163ec3)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:20:21.862668 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-c82d2578-bdc2-4bef-8530-ff207f593ba9 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:20:21.873083 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1200/2393] 10.222.0.52 () {60 vars in 1482 bytes} [Wed Jul 13 22:20:21 2022] PUT /identity/v3/projects/4f783cd7b91c459787f75abc3f0ccc5c/users/5eaa6a590eb14e8c80fde3fe19cbb8c4/roles/44efba2579f342df8a0789c59d163ec3 => generated 0 bytes in 33 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 22:20:21.883930 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5d4c3a-2722-4359-93e9-40f10fed939f None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:20:21.884220 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5d4c3a-2722-4359-93e9-40f10fed939f None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:20:21.884492 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-aa5d4c3a-2722-4359-93e9-40f10fed939f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:20:21.911644 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-aa5d4c3a-2722-4359-93e9-40f10fed939f None None] MFA Rules not processed for user `5eaa6a590eb14e8c80fde3fe19cbb8c4`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 22:20:21.912845 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-aa5d4c3a-2722-4359-93e9-40f10fed939f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:20:21.981744 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1194/2394] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jul 13 22:20:21 2022] POST /identity/v3/auth/tokens => generated 3307 bytes in 99 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 13 22:20:21.994108 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82d2578-bdc2-4bef-8530-ff207f593ba9 admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:20:21.997082 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-c82d2578-bdc2-4bef-8530-ff207f593ba9 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:20:22.061903 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c82d2578-bdc2-4bef-8530-ff207f593ba9 admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:20:22.065959 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9110c8-7640-475a-807b-a18e4909adbe service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc77e1ea1ada4700b809beb1c980c5e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417e3e2f1333481795a089bef2e24bea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:20:22.066669 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9110c8-7640-475a-807b-a18e4909adbe service cinder] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:20:22.067006 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9110c8-7640-475a-807b-a18e4909adbe service cinder] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:20:22.067202 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f9110c8-7640-475a-807b-a18e4909adbe service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:20:22.086501 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=95,keystone:INSERT=5 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:20:22.127993 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9110c8-7640-475a-807b-a18e4909adbe service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:20:22.129481 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5f9110c8-7640-475a-807b-a18e4909adbe service cinder] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:20:22.138657 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1201/2395] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jul 13 22:20:21 2022] GET /identity/v3/auth/tokens => generated 3238 bytes in 146 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 22:20:25.214245 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4c4881-114b-4705-a84d-72724aa1385e None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:20:25.214761 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4c4881-114b-4705-a84d-72724aa1385e None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:20:25.214855 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be4c4881-114b-4705-a84d-72724aa1385e None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:20:25.243506 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-be4c4881-114b-4705-a84d-72724aa1385e None None] MFA Rules not processed for user `b866bacdafae4db6a32714c07c56515f`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 22:20:25.244747 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-be4c4881-114b-4705-a84d-72724aa1385e None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:20:25.309813 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1195/2396] 10.222.0.52 () {58 vars in 1009 bytes} [Wed Jul 13 22:20:25 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 22:20:25.361888 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9110c8-7640-475a-807b-a18e4909adbe service cinder] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:20:25.363438 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-5f9110c8-7640-475a-807b-a18e4909adbe service cinder] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:20:25.437787 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5f9110c8-7640-475a-807b-a18e4909adbe service cinder] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:20:25.443771 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f928e40a-b0c3-4160-b6e1-75b2599022f4 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eb1525233ba48c99b15d4cb75db4822', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417e3e2f1333481795a089bef2e24bea', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:20:25.444431 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f928e40a-b0c3-4160-b6e1-75b2599022f4 service swift] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:20:25.444710 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f928e40a-b0c3-4160-b6e1-75b2599022f4 service swift] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:20:25.444984 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f928e40a-b0c3-4160-b6e1-75b2599022f4 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:20:25.453609 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f928e40a-b0c3-4160-b6e1-75b2599022f4 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:20:25.455424 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f928e40a-b0c3-4160-b6e1-75b2599022f4 service swift] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:20:25.463674 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1202/2397] 10.222.0.52 () {60 vars in 1384 bytes} [Wed Jul 13 22:20:25 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 102 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 13 22:20:25.745105 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4c4881-114b-4705-a84d-72724aa1385e None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:20:25.758005 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=100 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:20:25.823001 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be4c4881-114b-4705-a84d-72724aa1385e None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:20:25.828232 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be828f4a-ad5e-4a30-b2a0-cb86d2822e76 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:20:25.829123 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be828f4a-ad5e-4a30-b2a0-cb86d2822e76 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:20:25.829403 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be828f4a-ad5e-4a30-b2a0-cb86d2822e76 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:20:25.829757 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be828f4a-ad5e-4a30-b2a0-cb86d2822e76 admin admin] PATH_INFO: `/v3/users/8bf3c57f7c7b4ccfa71827e5f6b32acb` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:20:25.831135 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be828f4a-ad5e-4a30-b2a0-cb86d2822e76 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8bf3c57f7c7b4ccfa71827e5f6b32acb)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:20:25.833152 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be828f4a-ad5e-4a30-b2a0-cb86d2822e76 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:20:25.902259 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-be828f4a-ad5e-4a30-b2a0-cb86d2822e76 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8bf3c57f7c7b4ccfa71827e5f6b32acb'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:20:25.915727 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-be828f4a-ad5e-4a30-b2a0-cb86d2822e76 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8bf3c57f7c7b4ccfa71827e5f6b32acb'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:20:25.934251 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-be828f4a-ad5e-4a30-b2a0-cb86d2822e76 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8bf3c57f7c7b4ccfa71827e5f6b32acb'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:20:25.935058 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-be828f4a-ad5e-4a30-b2a0-cb86d2822e76 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8bf3c57f7c7b4ccfa71827e5f6b32acb'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:20:25.935706 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-be828f4a-ad5e-4a30-b2a0-cb86d2822e76 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8bf3c57f7c7b4ccfa71827e5f6b32acb'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:20:25.955901 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1196/2398] 10.222.0.52 () {58 vars in 1223 bytes} [Wed Jul 13 22:20:25 2022] DELETE /identity/v3/users/8bf3c57f7c7b4ccfa71827e5f6b32acb => generated 0 bytes in 211 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 22:20:25.963191 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da3bc50-6488-457a-b9a3-c51e6fdd9b5c None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:20:25.963372 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da3bc50-6488-457a-b9a3-c51e6fdd9b5c None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:20:25.963816 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5da3bc50-6488-457a-b9a3-c51e6fdd9b5c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:20:25.992016 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-5da3bc50-6488-457a-b9a3-c51e6fdd9b5c None None] MFA Rules not processed for user `cc99c93da5074cf5a48bc2aa9a92c5e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 22:20:25.993251 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-5da3bc50-6488-457a-b9a3-c51e6fdd9b5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:20:26.082034 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1203/2399] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jul 13 22:20:25 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 122 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 22:20:26.092709 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be828f4a-ad5e-4a30-b2a0-cb86d2822e76 admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:20:26.094428 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-be828f4a-ad5e-4a30-b2a0-cb86d2822e76 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:20:26.219757 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be828f4a-ad5e-4a30-b2a0-cb86d2822e76 admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:20:26.226830 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e47f2ac-a950-4555-b47d-9048792f7d33 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '847ec9a1bd4d4dec804318a5c845895a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417e3e2f1333481795a089bef2e24bea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:20:26.228579 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e47f2ac-a950-4555-b47d-9048792f7d33 service neutron] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:20:26.228917 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e47f2ac-a950-4555-b47d-9048792f7d33 service neutron] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:20:26.229266 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8e47f2ac-a950-4555-b47d-9048792f7d33 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:20:26.240442 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e47f2ac-a950-4555-b47d-9048792f7d33 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:20:26.242090 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-8e47f2ac-a950-4555-b47d-9048792f7d33 service neutron] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:20:26.257425 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1197/2400] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jul 13 22:20:26 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 165 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 22:20:26.601665 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da3bc50-6488-457a-b9a3-c51e6fdd9b5c None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:20:26.604138 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-5da3bc50-6488-457a-b9a3-c51e6fdd9b5c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:20:26.637783 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5da3bc50-6488-457a-b9a3-c51e6fdd9b5c None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:20:26.644581 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0621e65b-163d-41b2-9d59-b8e07a01eeb7 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:20:26.645538 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0621e65b-163d-41b2-9d59-b8e07a01eeb7 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:20:26.645877 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0621e65b-163d-41b2-9d59-b8e07a01eeb7 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:20:26.646208 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0621e65b-163d-41b2-9d59-b8e07a01eeb7 admin admin] PATH_INFO: `/v3/projects/4b816123ed564cdca5b0b35454176fa1` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:20:26.647497 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0621e65b-163d-41b2-9d59-b8e07a01eeb7 admin admin] RBAC: Authorizing `identity:delete_project(project_id=4b816123ed564cdca5b0b35454176fa1)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:20:26.649545 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0621e65b-163d-41b2-9d59-b8e07a01eeb7 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:20:26.714046 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=97,keystone:DELETE=3 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:20:26.718413 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-0621e65b-163d-41b2-9d59-b8e07a01eeb7 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4b816123ed564cdca5b0b35454176fa1', 'request_id': 'req-0621e65b-163d-41b2-9d59-b8e07a01eeb7'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:20:26.728007 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-0621e65b-163d-41b2-9d59-b8e07a01eeb7 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4b816123ed564cdca5b0b35454176fa1', 'request_id': 'req-0621e65b-163d-41b2-9d59-b8e07a01eeb7'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:20:26.734680 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-0621e65b-163d-41b2-9d59-b8e07a01eeb7 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4b816123ed564cdca5b0b35454176fa1', 'request_id': 'req-0621e65b-163d-41b2-9d59-b8e07a01eeb7'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:20:26.752486 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-0621e65b-163d-41b2-9d59-b8e07a01eeb7 admin admin] The token cache is being invalidate because project 4b816123ed564cdca5b0b35454176fa1 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80381) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 22:20:26.752868 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-0621e65b-163d-41b2-9d59-b8e07a01eeb7 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:20:26.753575 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-0621e65b-163d-41b2-9d59-b8e07a01eeb7 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:20:26.755459 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1204/2401] 10.222.0.52 () {58 vars in 1232 bytes} [Wed Jul 13 22:20:26 2022] DELETE /identity/v3/projects/4b816123ed564cdca5b0b35454176fa1 => generated 0 bytes in 154 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 22:20:26.759424 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e47f2ac-a950-4555-b47d-9048792f7d33 service neutron] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:20:26.769866 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-8e47f2ac-a950-4555-b47d-9048792f7d33 service neutron] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:20:26.775761 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:20:26.836145 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1bbef5d-0746-4b46-822a-7ce3ceab5b17 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:20:26.837384 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bbef5d-0746-4b46-822a-7ce3ceab5b17 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:20:26.837669 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bbef5d-0746-4b46-822a-7ce3ceab5b17 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:20:26.837947 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1bbef5d-0746-4b46-822a-7ce3ceab5b17 admin admin] PATH_INFO: `/v3/users/5eaa6a590eb14e8c80fde3fe19cbb8c4` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:20:26.839248 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1bbef5d-0746-4b46-822a-7ce3ceab5b17 admin admin] RBAC: Authorizing `identity:delete_user(user_id=5eaa6a590eb14e8c80fde3fe19cbb8c4)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:20:26.841224 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a1bbef5d-0746-4b46-822a-7ce3ceab5b17 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:20:26.901082 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-a1bbef5d-0746-4b46-822a-7ce3ceab5b17 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '5eaa6a590eb14e8c80fde3fe19cbb8c4'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:20:26.912708 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-a1bbef5d-0746-4b46-822a-7ce3ceab5b17 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '5eaa6a590eb14e8c80fde3fe19cbb8c4'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:20:26.927427 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-a1bbef5d-0746-4b46-822a-7ce3ceab5b17 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '5eaa6a590eb14e8c80fde3fe19cbb8c4'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:20:26.928182 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-a1bbef5d-0746-4b46-822a-7ce3ceab5b17 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '5eaa6a590eb14e8c80fde3fe19cbb8c4'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:20:26.928873 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-a1bbef5d-0746-4b46-822a-7ce3ceab5b17 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '5eaa6a590eb14e8c80fde3fe19cbb8c4'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:20:26.944280 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1198/2402] 10.222.0.52 () {58 vars in 1223 bytes} [Wed Jul 13 22:20:26 2022] DELETE /identity/v3/users/5eaa6a590eb14e8c80fde3fe19cbb8c4 => generated 0 bytes in 185 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 13 22:20:27.284760 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0621e65b-163d-41b2-9d59-b8e07a01eeb7 admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:20:27.288259 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-0621e65b-163d-41b2-9d59-b8e07a01eeb7 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:20:27.311660 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0621e65b-163d-41b2-9d59-b8e07a01eeb7 admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:20:27.423514 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a18a5195-3145-4e68-a4e4-121acafb7bcb admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:20:27.424692 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18a5195-3145-4e68-a4e4-121acafb7bcb admin admin] REQUEST_METHOD: `DELETE` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:20:27.425043 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18a5195-3145-4e68-a4e4-121acafb7bcb admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:20:27.425396 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a18a5195-3145-4e68-a4e4-121acafb7bcb admin admin] PATH_INFO: `/v3/projects/4f783cd7b91c459787f75abc3f0ccc5c` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:20:27.426881 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a18a5195-3145-4e68-a4e4-121acafb7bcb admin admin] RBAC: Authorizing `identity:delete_project(project_id=4f783cd7b91c459787f75abc3f0ccc5c)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:20:27.428874 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a18a5195-3145-4e68-a4e4-121acafb7bcb admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:20:27.499489 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-a18a5195-3145-4e68-a4e4-121acafb7bcb admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '4f783cd7b91c459787f75abc3f0ccc5c', 'request_id': 'req-a18a5195-3145-4e68-a4e4-121acafb7bcb'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:20:27.506672 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-a18a5195-3145-4e68-a4e4-121acafb7bcb admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '4f783cd7b91c459787f75abc3f0ccc5c', 'request_id': 'req-a18a5195-3145-4e68-a4e4-121acafb7bcb'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:20:27.517669 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-a18a5195-3145-4e68-a4e4-121acafb7bcb admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '4f783cd7b91c459787f75abc3f0ccc5c', 'request_id': 'req-a18a5195-3145-4e68-a4e4-121acafb7bcb'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:20:27.538805 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-a18a5195-3145-4e68-a4e4-121acafb7bcb admin admin] The token cache is being invalidate because project 4f783cd7b91c459787f75abc3f0ccc5c was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80381) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 22:20:27.539666 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-a18a5195-3145-4e68-a4e4-121acafb7bcb admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:20:27.540525 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-a18a5195-3145-4e68-a4e4-121acafb7bcb admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:20:27.542870 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1205/2403] 10.222.0.52 () {58 vars in 1232 bytes} [Wed Jul 13 22:20:27 2022] DELETE /identity/v3/projects/4f783cd7b91c459787f75abc3f0ccc5c => generated 0 bytes in 259 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 22:20:27.555630 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad519b15-70f1-482d-a977-33ff2073da4b None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:20:27.556088 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad519b15-70f1-482d-a977-33ff2073da4b None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:20:27.556214 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-ad519b15-70f1-482d-a977-33ff2073da4b None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:20:27.580468 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-ad519b15-70f1-482d-a977-33ff2073da4b None None] MFA Rules not processed for user `cc99c93da5074cf5a48bc2aa9a92c5e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 22:20:27.581696 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-ad519b15-70f1-482d-a977-33ff2073da4b None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:20:27.656964 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1199/2404] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jul 13 22:20:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 110 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 22:20:27.666864 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22ee8fa-f11e-48b9-a075-8cc432251aa3 None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:20:27.667417 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22ee8fa-f11e-48b9-a075-8cc432251aa3 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:20:27.667417 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c22ee8fa-f11e-48b9-a075-8cc432251aa3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:20:27.693790 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-c22ee8fa-f11e-48b9-a075-8cc432251aa3 None None] MFA Rules not processed for user `cc99c93da5074cf5a48bc2aa9a92c5e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 22:20:27.695065 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-c22ee8fa-f11e-48b9-a075-8cc432251aa3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:20:27.707589 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1206/2405] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jul 13 22:20:27 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 42 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 22:20:27.708600 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad519b15-70f1-482d-a977-33ff2073da4b None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:20:27.724431 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-ad519b15-70f1-482d-a977-33ff2073da4b None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:20:27.729659 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5bc06e-9dab-4baa-b0e2-e0005b61243d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:20:27.730514 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5bc06e-9dab-4baa-b0e2-e0005b61243d admin admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:20:27.730803 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5bc06e-9dab-4baa-b0e2-e0005b61243d admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:20:27.731076 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c5bc06e-9dab-4baa-b0e2-e0005b61243d admin admin] PATH_INFO: `/v3/domains` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:20:27.731818 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5bc06e-9dab-4baa-b0e2-e0005b61243d admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 22:20:27.732123 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5bc06e-9dab-4baa-b0e2-e0005b61243d admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:20:27.733780 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7c5bc06e-9dab-4baa-b0e2-e0005b61243d admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:20:27.752914 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1200/2406] 10.222.0.52 () {58 vars in 1152 bytes} [Wed Jul 13 22:20:27 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 44 msecs (HTTP/1.1 200) 5 headers in 177 bytes (2 switches on core 0) Jul 13 22:20:27.757329 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22ee8fa-f11e-48b9-a075-8cc432251aa3 None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:20:27.771319 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c22ee8fa-f11e-48b9-a075-8cc432251aa3 None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:20:27.777502 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66620d59-c9eb-4053-b56d-cd7c3969b62d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:20:27.778809 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66620d59-c9eb-4053-b56d-cd7c3969b62d admin admin] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:20:27.779327 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66620d59-c9eb-4053-b56d-cd7c3969b62d admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:20:27.780060 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-66620d59-c9eb-4053-b56d-cd7c3969b62d admin admin] PATH_INFO: `/v3/projects` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:20:27.781150 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66620d59-c9eb-4053-b56d-cd7c3969b62d admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:20:27.784223 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-66620d59-c9eb-4053-b56d-cd7c3969b62d admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:20:27.826692 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1207/2407] 10.222.0.52 () {60 vars in 1152 bytes} [Wed Jul 13 22:20:27 2022] POST /identity/v3/projects => generated 373 bytes in 70 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 22:20:27.903973 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5bc06e-9dab-4baa-b0e2-e0005b61243d admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:20:27.920120 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c5bc06e-9dab-4baa-b0e2-e0005b61243d admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:20:27.936194 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=89,keystone:DELETE=10,keystone:INSERT=1 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:20:27.994948 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0365e28b-4bf9-440f-8c5f-e1cccb2fb6e2 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:20:27.995648 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0365e28b-4bf9-440f-8c5f-e1cccb2fb6e2 admin admin] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:20:27.995928 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0365e28b-4bf9-440f-8c5f-e1cccb2fb6e2 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:20:27.996205 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0365e28b-4bf9-440f-8c5f-e1cccb2fb6e2 admin admin] PATH_INFO: `/v3/users` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:20:27.996924 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0365e28b-4bf9-440f-8c5f-e1cccb2fb6e2 admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:20:27.998991 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0365e28b-4bf9-440f-8c5f-e1cccb2fb6e2 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:20:28.029307 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1201/2408] 10.222.0.52 () {60 vars in 1143 bytes} [Wed Jul 13 22:20:27 2022] POST /identity/v3/users => generated 287 bytes in 123 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 22:20:28.030771 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66620d59-c9eb-4053-b56d-cd7c3969b62d admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:20:28.047127 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-66620d59-c9eb-4053-b56d-cd7c3969b62d admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:20:28.052194 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5300eb-7fa6-4ffb-a80b-9783f7f03655 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:20:28.052903 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5300eb-7fa6-4ffb-a80b-9783f7f03655 admin admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:20:28.053180 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5300eb-7fa6-4ffb-a80b-9783f7f03655 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:20:28.053477 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-be5300eb-7fa6-4ffb-a80b-9783f7f03655 admin admin] PATH_INFO: `/v3/roles` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:20:28.054230 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be5300eb-7fa6-4ffb-a80b-9783f7f03655 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:20:28.055970 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-be5300eb-7fa6-4ffb-a80b-9783f7f03655 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:20:28.071849 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1208/2409] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jul 13 22:20:28 2022] GET /identity/v3/roles => generated 1420 bytes in 38 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 22:20:28.073124 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0365e28b-4bf9-440f-8c5f-e1cccb2fb6e2 admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:20:28.085763 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0365e28b-4bf9-440f-8c5f-e1cccb2fb6e2 admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:20:28.091454 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dc755c2b-4a92-4336-bf2f-998afd24659d admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:20:28.092171 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc755c2b-4a92-4336-bf2f-998afd24659d admin admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:20:28.092451 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc755c2b-4a92-4336-bf2f-998afd24659d admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:20:28.092723 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dc755c2b-4a92-4336-bf2f-998afd24659d admin admin] PATH_INFO: `/v3/roles` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:20:28.093484 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc755c2b-4a92-4336-bf2f-998afd24659d admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:20:28.095259 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dc755c2b-4a92-4336-bf2f-998afd24659d admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:20:28.109173 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1202/2410] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jul 13 22:20:28 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 22:20:28.112723 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5300eb-7fa6-4ffb-a80b-9783f7f03655 admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:20:28.123682 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-be5300eb-7fa6-4ffb-a80b-9783f7f03655 admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:20:28.128363 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55aad97-d677-4601-b3d0-29e41dc8721c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:20:28.129254 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55aad97-d677-4601-b3d0-29e41dc8721c admin admin] REQUEST_METHOD: `PUT` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:20:28.129528 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55aad97-d677-4601-b3d0-29e41dc8721c admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:20:28.129798 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a55aad97-d677-4601-b3d0-29e41dc8721c admin admin] PATH_INFO: `/v3/projects/18736eddcf0348e6a2dc039e5be13374/users/8e20d279bd08449ca2e9acb114572a12/roles/2a5e86976394498abb6310697cdb3f8b` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:20:28.149710 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55aad97-d677-4601-b3d0-29e41dc8721c admin admin] RBAC: Authorizing `identity:create_grant(project_id=18736eddcf0348e6a2dc039e5be13374, user_id=8e20d279bd08449ca2e9acb114572a12, role_id=2a5e86976394498abb6310697cdb3f8b)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:20:28.159303 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-a55aad97-d677-4601-b3d0-29e41dc8721c admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:20:28.203508 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:DELETE=11,keystone:SELECT=85,keystone:INSERT=4 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:20:28.209352 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1209/2411] 10.222.0.52 () {60 vars in 1482 bytes} [Wed Jul 13 22:20:28 2022] PUT /identity/v3/projects/18736eddcf0348e6a2dc039e5be13374/users/8e20d279bd08449ca2e9acb114572a12/roles/2a5e86976394498abb6310697cdb3f8b => generated 0 bytes in 97 msecs (HTTP/1.1 204) 3 headers in 132 bytes (1 switches on core 0) Jul 13 22:20:28.220064 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b8e08e-9779-42ad-b343-ed20d3c641f0 None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:20:28.220451 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b8e08e-9779-42ad-b343-ed20d3c641f0 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:20:28.220652 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-48b8e08e-9779-42ad-b343-ed20d3c641f0 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:20:28.255146 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-48b8e08e-9779-42ad-b343-ed20d3c641f0 None None] MFA Rules not processed for user `8e20d279bd08449ca2e9acb114572a12`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 22:20:28.256516 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-48b8e08e-9779-42ad-b343-ed20d3c641f0 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:20:28.342959 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1203/2412] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jul 13 22:20:28 2022] POST /identity/v3/auth/tokens => generated 3240 bytes in 124 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 22:20:28.354706 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55aad97-d677-4601-b3d0-29e41dc8721c admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:20:28.356993 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-a55aad97-d677-4601-b3d0-29e41dc8721c admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:20:28.435825 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a55aad97-d677-4601-b3d0-29e41dc8721c admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:20:28.440386 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-18ee14f8-75e5-49cd-9362-bf5b27a9a7ed service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc77e1ea1ada4700b809beb1c980c5e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417e3e2f1333481795a089bef2e24bea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:20:28.441135 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ee14f8-75e5-49cd-9362-bf5b27a9a7ed service cinder] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:20:28.441511 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ee14f8-75e5-49cd-9362-bf5b27a9a7ed service cinder] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:20:28.441859 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-18ee14f8-75e5-49cd-9362-bf5b27a9a7ed service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:20:28.454692 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ee14f8-75e5-49cd-9362-bf5b27a9a7ed service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:20:28.456165 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-18ee14f8-75e5-49cd-9362-bf5b27a9a7ed service cinder] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:20:28.466478 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1210/2413] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jul 13 22:20:28 2022] GET /identity/v3/auth/tokens => generated 3240 bytes in 112 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 22:20:31.783306 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b8e08e-9779-42ad-b343-ed20d3c641f0 None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:20:31.849278 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-48b8e08e-9779-42ad-b343-ed20d3c641f0 None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:20:31.853912 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b68dae-bba0-4778-bfa8-52e4dd449ed3 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:20:31.854831 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b68dae-bba0-4778-bfa8-52e4dd449ed3 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:20:31.855113 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b68dae-bba0-4778-bfa8-52e4dd449ed3 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:20:31.855388 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f8b68dae-bba0-4778-bfa8-52e4dd449ed3 admin admin] PATH_INFO: `/v3/users/8e20d279bd08449ca2e9acb114572a12` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:20:31.856595 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b68dae-bba0-4778-bfa8-52e4dd449ed3 admin admin] RBAC: Authorizing `identity:delete_user(user_id=8e20d279bd08449ca2e9acb114572a12)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:20:31.858600 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f8b68dae-bba0-4778-bfa8-52e4dd449ed3 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:20:31.889600 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=93,keystone:INSERT=3,keystone:DELETE=4 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:20:31.911532 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-f8b68dae-bba0-4778-bfa8-52e4dd449ed3 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '8e20d279bd08449ca2e9acb114572a12'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:20:31.920365 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-f8b68dae-bba0-4778-bfa8-52e4dd449ed3 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '8e20d279bd08449ca2e9acb114572a12'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:20:31.935061 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-f8b68dae-bba0-4778-bfa8-52e4dd449ed3 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '8e20d279bd08449ca2e9acb114572a12'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:20:31.935756 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-f8b68dae-bba0-4778-bfa8-52e4dd449ed3 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '8e20d279bd08449ca2e9acb114572a12'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:20:31.936601 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-f8b68dae-bba0-4778-bfa8-52e4dd449ed3 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '8e20d279bd08449ca2e9acb114572a12'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:20:31.953680 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1204/2414] 10.222.0.52 () {58 vars in 1223 bytes} [Wed Jul 13 22:20:31 2022] DELETE /identity/v3/users/8e20d279bd08449ca2e9acb114572a12 => generated 0 bytes in 168 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 22:20:31.955731 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad08762-23a7-4502-8ca6-35ac4700f2c1 None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:20:31.956238 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad08762-23a7-4502-8ca6-35ac4700f2c1 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:20:31.956424 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1ad08762-23a7-4502-8ca6-35ac4700f2c1 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:20:31.974729 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-1ad08762-23a7-4502-8ca6-35ac4700f2c1 None None] MFA Rules not processed for user `cc99c93da5074cf5a48bc2aa9a92c5e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 22:20:31.976051 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-1ad08762-23a7-4502-8ca6-35ac4700f2c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:20:32.052565 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1211/2415] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jul 13 22:20:31 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 98 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 22:20:32.062463 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b68dae-bba0-4778-bfa8-52e4dd449ed3 admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:20:32.064484 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-f8b68dae-bba0-4778-bfa8-52e4dd449ed3 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:20:32.139868 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f8b68dae-bba0-4778-bfa8-52e4dd449ed3 admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:20:32.148681 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1d7fda83-e6e4-4c8b-91c1-064e032ad80d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '847ec9a1bd4d4dec804318a5c845895a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417e3e2f1333481795a089bef2e24bea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:20:32.149390 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7fda83-e6e4-4c8b-91c1-064e032ad80d service neutron] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:20:32.149727 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7fda83-e6e4-4c8b-91c1-064e032ad80d service neutron] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:20:32.150068 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1d7fda83-e6e4-4c8b-91c1-064e032ad80d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:20:32.162909 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7fda83-e6e4-4c8b-91c1-064e032ad80d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:20:32.164940 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-1d7fda83-e6e4-4c8b-91c1-064e032ad80d service neutron] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:20:32.179845 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1205/2416] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jul 13 22:20:32 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 118 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 22:20:32.556873 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad08762-23a7-4502-8ca6-35ac4700f2c1 None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:20:32.558530 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-1ad08762-23a7-4502-8ca6-35ac4700f2c1 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:20:32.595301 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1ad08762-23a7-4502-8ca6-35ac4700f2c1 None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:20:32.607883 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7ccbb710-24b1-4b64-a1f9-0af7aa4e253e admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:20:32.608799 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccbb710-24b1-4b64-a1f9-0af7aa4e253e admin admin] REQUEST_METHOD: `DELETE` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:20:32.609148 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccbb710-24b1-4b64-a1f9-0af7aa4e253e admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:20:32.609425 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7ccbb710-24b1-4b64-a1f9-0af7aa4e253e admin admin] PATH_INFO: `/v3/projects/18736eddcf0348e6a2dc039e5be13374` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:20:32.612878 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ccbb710-24b1-4b64-a1f9-0af7aa4e253e admin admin] RBAC: Authorizing `identity:delete_project(project_id=18736eddcf0348e6a2dc039e5be13374)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:20:32.617953 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7ccbb710-24b1-4b64-a1f9-0af7aa4e253e admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:20:32.699188 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-7ccbb710-24b1-4b64-a1f9-0af7aa4e253e admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '18736eddcf0348e6a2dc039e5be13374', 'request_id': 'req-7ccbb710-24b1-4b64-a1f9-0af7aa4e253e'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:20:32.706776 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-7ccbb710-24b1-4b64-a1f9-0af7aa4e253e admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '18736eddcf0348e6a2dc039e5be13374', 'request_id': 'req-7ccbb710-24b1-4b64-a1f9-0af7aa4e253e'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:20:32.713471 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-7ccbb710-24b1-4b64-a1f9-0af7aa4e253e admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '18736eddcf0348e6a2dc039e5be13374', 'request_id': 'req-7ccbb710-24b1-4b64-a1f9-0af7aa4e253e'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:20:32.733541 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-7ccbb710-24b1-4b64-a1f9-0af7aa4e253e admin admin] The token cache is being invalidate because project 18736eddcf0348e6a2dc039e5be13374 was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80381) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 22:20:32.733541 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-7ccbb710-24b1-4b64-a1f9-0af7aa4e253e admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:20:32.734340 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-7ccbb710-24b1-4b64-a1f9-0af7aa4e253e admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:20:32.736614 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1212/2417] 10.222.0.52 () {58 vars in 1232 bytes} [Wed Jul 13 22:20:32 2022] DELETE /identity/v3/projects/18736eddcf0348e6a2dc039e5be13374 => generated 0 bytes in 180 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 22:20:32.741662 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a08abf9-8b0f-408f-8c81-a8066be17fad None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:20:32.742255 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a08abf9-8b0f-408f-8c81-a8066be17fad None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:20:32.742635 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-1a08abf9-8b0f-408f-8c81-a8066be17fad None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:20:32.761916 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-1a08abf9-8b0f-408f-8c81-a8066be17fad None None] MFA Rules not processed for user `cc99c93da5074cf5a48bc2aa9a92c5e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 22:20:32.763440 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-1a08abf9-8b0f-408f-8c81-a8066be17fad None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:20:32.856314 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1206/2418] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jul 13 22:20:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 116 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 22:20:32.864806 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3b687b-bff7-41aa-aa49-8ae99e17a9f7 None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:20:32.865102 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3b687b-bff7-41aa-aa49-8ae99e17a9f7 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:20:32.865384 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7c3b687b-bff7-41aa-aa49-8ae99e17a9f7 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:20:32.886607 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-7c3b687b-bff7-41aa-aa49-8ae99e17a9f7 None None] MFA Rules not processed for user `cc99c93da5074cf5a48bc2aa9a92c5e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 22:20:32.888181 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-7c3b687b-bff7-41aa-aa49-8ae99e17a9f7 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:20:32.898108 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1213/2419] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jul 13 22:20:32 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 22:20:32.902436 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a08abf9-8b0f-408f-8c81-a8066be17fad None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:20:32.914162 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-1a08abf9-8b0f-408f-8c81-a8066be17fad None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:20:32.924009 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0213f2-d430-4307-a9ce-4e5f26aed0ca admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:20:32.924717 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0213f2-d430-4307-a9ce-4e5f26aed0ca admin admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:20:32.925173 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0213f2-d430-4307-a9ce-4e5f26aed0ca admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:20:32.925559 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5c0213f2-d430-4307-a9ce-4e5f26aed0ca admin admin] PATH_INFO: `/v3/domains` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:20:32.926317 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0213f2-d430-4307-a9ce-4e5f26aed0ca admin admin] RBAC: Adding query filter params (name=Default) {{(pid=80382) _extract_filter_values /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:167}} Jul 13 22:20:32.926654 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0213f2-d430-4307-a9ce-4e5f26aed0ca admin admin] RBAC: Authorizing `identity:list_domains()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:20:32.928376 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5c0213f2-d430-4307-a9ce-4e5f26aed0ca admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:20:32.947432 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1207/2420] 10.222.0.52 () {58 vars in 1152 bytes} [Wed Jul 13 22:20:32 2022] GET /identity/v3/domains?name=Default => generated 308 bytes in 42 msecs (HTTP/1.1 200) 5 headers in 177 bytes (1 switches on core 0) Jul 13 22:20:32.948953 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3b687b-bff7-41aa-aa49-8ae99e17a9f7 None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:20:32.960332 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7c3b687b-bff7-41aa-aa49-8ae99e17a9f7 None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:20:32.966207 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39fd9e97-8dc5-47f1-9ae5-e728af824b58 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:20:32.967355 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fd9e97-8dc5-47f1-9ae5-e728af824b58 admin admin] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:20:32.967818 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fd9e97-8dc5-47f1-9ae5-e728af824b58 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:20:32.968439 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-39fd9e97-8dc5-47f1-9ae5-e728af824b58 admin admin] PATH_INFO: `/v3/projects` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:20:32.969573 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39fd9e97-8dc5-47f1-9ae5-e728af824b58 admin admin] RBAC: Authorizing `identity:create_project()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:20:32.972324 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-39fd9e97-8dc5-47f1-9ae5-e728af824b58 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:20:32.996347 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=91,keystone:DELETE=7,keystone:INSERT=2 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:20:33.010329 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1214/2421] 10.222.0.52 () {60 vars in 1152 bytes} [Wed Jul 13 22:20:32 2022] POST /identity/v3/projects => generated 385 bytes in 62 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 22:20:33.012010 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0213f2-d430-4307-a9ce-4e5f26aed0ca admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:20:33.026241 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5c0213f2-d430-4307-a9ce-4e5f26aed0ca admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:20:33.075550 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:DELETE=6,keystone:SELECT=93,keystone:INSERT=1 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:20:33.111412 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380c3968-62e0-40a9-9e50-760c6595d51c admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:20:33.112011 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380c3968-62e0-40a9-9e50-760c6595d51c admin admin] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:20:33.112291 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380c3968-62e0-40a9-9e50-760c6595d51c admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:20:33.112564 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-380c3968-62e0-40a9-9e50-760c6595d51c admin admin] PATH_INFO: `/v3/users` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:20:33.113587 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-380c3968-62e0-40a9-9e50-760c6595d51c admin admin] RBAC: Authorizing `identity:create_user()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:20:33.115776 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-380c3968-62e0-40a9-9e50-760c6595d51c admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:20:33.143217 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1208/2422] 10.222.0.52 () {60 vars in 1143 bytes} [Wed Jul 13 22:20:33 2022] POST /identity/v3/users => generated 293 bytes in 132 msecs (HTTP/1.1 201) 5 headers in 182 bytes (1 switches on core 0) Jul 13 22:20:33.147460 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39fd9e97-8dc5-47f1-9ae5-e728af824b58 admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:20:33.159634 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-39fd9e97-8dc5-47f1-9ae5-e728af824b58 admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:20:33.165271 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449a2aa2-ef4b-4ea2-854b-73f2f08f24ec admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:20:33.166121 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449a2aa2-ef4b-4ea2-854b-73f2f08f24ec admin admin] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:20:33.166404 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449a2aa2-ef4b-4ea2-854b-73f2f08f24ec admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:20:33.166715 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-449a2aa2-ef4b-4ea2-854b-73f2f08f24ec admin admin] PATH_INFO: `/v3/roles` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:20:33.167452 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-449a2aa2-ef4b-4ea2-854b-73f2f08f24ec admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:20:33.169084 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-449a2aa2-ef4b-4ea2-854b-73f2f08f24ec admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:20:33.182892 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1215/2423] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jul 13 22:20:33 2022] GET /identity/v3/roles => generated 1420 bytes in 36 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 22:20:33.187041 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380c3968-62e0-40a9-9e50-760c6595d51c admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:20:33.198990 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-380c3968-62e0-40a9-9e50-760c6595d51c admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:20:33.204273 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd85c0e3-dcb4-4e75-b0f1-57344b39a733 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:20:33.205500 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd85c0e3-dcb4-4e75-b0f1-57344b39a733 admin admin] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:20:33.205798 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd85c0e3-dcb4-4e75-b0f1-57344b39a733 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:20:33.206071 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd85c0e3-dcb4-4e75-b0f1-57344b39a733 admin admin] PATH_INFO: `/v3/roles` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:20:33.207009 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd85c0e3-dcb4-4e75-b0f1-57344b39a733 admin admin] RBAC: Authorizing `identity:list_roles()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:20:33.208617 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd85c0e3-dcb4-4e75-b0f1-57344b39a733 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:20:33.224982 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1209/2424] 10.222.0.52 () {58 vars in 1121 bytes} [Wed Jul 13 22:20:33 2022] GET /identity/v3/roles => generated 1420 bytes in 35 msecs (HTTP/1.1 200) 5 headers in 178 bytes (1 switches on core 0) Jul 13 22:20:33.227564 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449a2aa2-ef4b-4ea2-854b-73f2f08f24ec admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:20:33.240160 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-449a2aa2-ef4b-4ea2-854b-73f2f08f24ec admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:20:33.246210 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f769890b-b722-45d0-99c9-47ab2a785193 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:20:33.247553 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f769890b-b722-45d0-99c9-47ab2a785193 admin admin] REQUEST_METHOD: `PUT` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:20:33.247909 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f769890b-b722-45d0-99c9-47ab2a785193 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:20:33.248420 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f769890b-b722-45d0-99c9-47ab2a785193 admin admin] PATH_INFO: `/v3/projects/614cb1cb24bd4bc8b7c48e5f47b501ca/users/7fea6e72f4884a0cb3471ea6239551ba/roles/2a5e86976394498abb6310697cdb3f8b` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:20:33.277178 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f769890b-b722-45d0-99c9-47ab2a785193 admin admin] RBAC: Authorizing `identity:create_grant(project_id=614cb1cb24bd4bc8b7c48e5f47b501ca, user_id=7fea6e72f4884a0cb3471ea6239551ba, role_id=2a5e86976394498abb6310697cdb3f8b)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:20:33.282402 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f769890b-b722-45d0-99c9-47ab2a785193 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:20:33.300475 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1216/2425] 10.222.0.52 () {60 vars in 1482 bytes} [Wed Jul 13 22:20:33 2022] PUT /identity/v3/projects/614cb1cb24bd4bc8b7c48e5f47b501ca/users/7fea6e72f4884a0cb3471ea6239551ba/roles/2a5e86976394498abb6310697cdb3f8b => generated 0 bytes in 75 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 22:20:33.312109 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e3818b-32b4-4baf-adf5-f68644b4d4c3 None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:20:33.312397 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e3818b-32b4-4baf-adf5-f68644b4d4c3 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:20:33.313297 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a4e3818b-32b4-4baf-adf5-f68644b4d4c3 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:20:33.342736 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-a4e3818b-32b4-4baf-adf5-f68644b4d4c3 None None] MFA Rules not processed for user `7fea6e72f4884a0cb3471ea6239551ba`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 22:20:33.343989 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-a4e3818b-32b4-4baf-adf5-f68644b4d4c3 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:20:33.439704 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1210/2426] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jul 13 22:20:33 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 128 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 22:20:33.452026 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f769890b-b722-45d0-99c9-47ab2a785193 admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:20:33.454118 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-f769890b-b722-45d0-99c9-47ab2a785193 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:20:33.586471 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f769890b-b722-45d0-99c9-47ab2a785193 admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:20:33.589607 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-56b82b2e-8f2e-440b-abc6-d848c49d5e75 service cinder] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'bc77e1ea1ada4700b809beb1c980c5e1', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417e3e2f1333481795a089bef2e24bea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:20:33.590296 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b82b2e-8f2e-440b-abc6-d848c49d5e75 service cinder] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:20:33.590615 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b82b2e-8f2e-440b-abc6-d848c49d5e75 service cinder] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:20:33.590895 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-56b82b2e-8f2e-440b-abc6-d848c49d5e75 service cinder] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:20:33.603169 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b82b2e-8f2e-440b-abc6-d848c49d5e75 service cinder] RBAC: Authorizing `identity:validate_token()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:20:33.604928 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-56b82b2e-8f2e-440b-abc6-d848c49d5e75 service cinder] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:20:33.627346 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1217/2427] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jul 13 22:20:33 2022] GET /identity/v3/auth/tokens => generated 3252 bytes in 176 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 22:20:36.321132 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e3818b-32b4-4baf-adf5-f68644b4d4c3 None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:20:36.389871 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a4e3818b-32b4-4baf-adf5-f68644b4d4c3 None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:20:36.395139 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f86cb1-7225-40f7-9905-373e451ab619 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:20:36.396017 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f86cb1-7225-40f7-9905-373e451ab619 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:20:36.396282 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f86cb1-7225-40f7-9905-373e451ab619 admin admin] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:20:36.396554 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-61f86cb1-7225-40f7-9905-373e451ab619 admin admin] PATH_INFO: `/v3/users/7fea6e72f4884a0cb3471ea6239551ba` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:20:36.397759 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f86cb1-7225-40f7-9905-373e451ab619 admin admin] RBAC: Authorizing `identity:delete_user(user_id=7fea6e72f4884a0cb3471ea6239551ba)` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:20:36.399716 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-61f86cb1-7225-40f7-9905-373e451ab619 admin admin] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:20:36.447415 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=91,keystone:INSERT=3,keystone:DELETE=6 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:20:36.450047 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-61f86cb1-7225-40f7-9905-373e451ab619 admin admin] Invoking callback _user_callback for event identity user deleted for {'resource_info': '7fea6e72f4884a0cb3471ea6239551ba'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:20:36.470219 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-61f86cb1-7225-40f7-9905-373e451ab619 admin admin] Invoking callback _delete_app_creds_on_user_delete_callback for event identity user deleted for {'resource_info': '7fea6e72f4884a0cb3471ea6239551ba'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:20:36.486707 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-61f86cb1-7225-40f7-9905-373e451ab619 admin admin] Invoking callback _drop_token_cache for event identity user deleted for {'resource_info': '7fea6e72f4884a0cb3471ea6239551ba'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:20:36.487644 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-61f86cb1-7225-40f7-9905-373e451ab619 admin admin] Invoking callback _drop_receipt_cache for event identity user deleted for {'resource_info': '7fea6e72f4884a0cb3471ea6239551ba'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:20:36.488379 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.notifications [None req-61f86cb1-7225-40f7-9905-373e451ab619 admin admin] Invoking callback _on_user_delete for event identity user deleted for {'resource_info': '7fea6e72f4884a0cb3471ea6239551ba'} {{(pid=80382) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:20:36.507091 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1211/2428] 10.222.0.52 () {58 vars in 1223 bytes} [Wed Jul 13 22:20:36 2022] DELETE /identity/v3/users/7fea6e72f4884a0cb3471ea6239551ba => generated 0 bytes in 187 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 22:20:36.511704 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f26db07-5f5d-404b-a7fa-62256613dc90 None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:20:36.512235 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f26db07-5f5d-404b-a7fa-62256613dc90 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:20:36.512235 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f26db07-5f5d-404b-a7fa-62256613dc90 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:20:36.533988 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-3f26db07-5f5d-404b-a7fa-62256613dc90 None None] MFA Rules not processed for user `cc99c93da5074cf5a48bc2aa9a92c5e7`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 22:20:36.535344 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-3f26db07-5f5d-404b-a7fa-62256613dc90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:20:36.625333 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1218/2429] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jul 13 22:20:36 2022] POST /identity/v3/auth/tokens => generated 3229 bytes in 112 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 13 22:20:36.636013 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f86cb1-7225-40f7-9905-373e451ab619 admin admin] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:20:36.637733 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-61f86cb1-7225-40f7-9905-373e451ab619 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:20:36.725375 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-61f86cb1-7225-40f7-9905-373e451ab619 admin admin] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:20:36.732110 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-17688386-713b-4a1a-a5c3-d22a533f2c1d service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '847ec9a1bd4d4dec804318a5c845895a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417e3e2f1333481795a089bef2e24bea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:20:36.732809 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17688386-713b-4a1a-a5c3-d22a533f2c1d service neutron] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:20:36.733123 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17688386-713b-4a1a-a5c3-d22a533f2c1d service neutron] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:20:36.733403 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-17688386-713b-4a1a-a5c3-d22a533f2c1d service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:20:36.748907 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17688386-713b-4a1a-a5c3-d22a533f2c1d service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:20:36.751924 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-17688386-713b-4a1a-a5c3-d22a533f2c1d service neutron] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:20:36.763565 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1212/2430] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jul 13 22:20:36 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 128 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 22:20:37.057531 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f26db07-5f5d-404b-a7fa-62256613dc90 None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:20:37.059350 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-3f26db07-5f5d-404b-a7fa-62256613dc90 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:20:37.072432 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f26db07-5f5d-404b-a7fa-62256613dc90 None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:20:37.083034 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e193fd4a-3e90-46e1-a4a4-329b28727682 admin admin] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'cc99c93da5074cf5a48bc2aa9a92c5e7', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '0b902d068f594f05b4de544301553ded', 'project_domain_id': 'default', 'roles': ['reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:20:37.083915 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e193fd4a-3e90-46e1-a4a4-329b28727682 admin admin] REQUEST_METHOD: `DELETE` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:20:37.084134 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e193fd4a-3e90-46e1-a4a4-329b28727682 admin admin] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:20:37.084407 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-e193fd4a-3e90-46e1-a4a4-329b28727682 admin admin] PATH_INFO: `/v3/projects/614cb1cb24bd4bc8b7c48e5f47b501ca` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:20:37.091377 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e193fd4a-3e90-46e1-a4a4-329b28727682 admin admin] RBAC: Authorizing `identity:delete_project(project_id=614cb1cb24bd4bc8b7c48e5f47b501ca)` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:20:37.093322 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-e193fd4a-3e90-46e1-a4a4-329b28727682 admin admin] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:20:37.137585 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=96,keystone:INSERT=1,keystone:DELETE=3 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:20:37.144679 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-e193fd4a-3e90-46e1-a4a4-329b28727682 admin admin] Invoking callback _unset_default_project for event identity project deleted for {'resource_info': '614cb1cb24bd4bc8b7c48e5f47b501ca', 'request_id': 'req-e193fd4a-3e90-46e1-a4a4-329b28727682'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:20:37.151523 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-e193fd4a-3e90-46e1-a4a4-329b28727682 admin admin] Invoking callback _on_project_or_endpoint_delete for event identity project deleted for {'resource_info': '614cb1cb24bd4bc8b7c48e5f47b501ca', 'request_id': 'req-e193fd4a-3e90-46e1-a4a4-329b28727682'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:20:37.158069 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-e193fd4a-3e90-46e1-a4a4-329b28727682 admin admin] Invoking callback _project_callback for event identity project deleted for {'resource_info': '614cb1cb24bd4bc8b7c48e5f47b501ca', 'request_id': 'req-e193fd4a-3e90-46e1-a4a4-329b28727682'} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:20:37.174409 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-e193fd4a-3e90-46e1-a4a4-329b28727682 admin admin] The token cache is being invalidate because project 614cb1cb24bd4bc8b7c48e5f47b501ca was deleted. Authorization will be recalculated and enforced accordingly the next time users authenticate or validate a token. {{(pid=80381) invalidate_token_cache_notification /opt/stack/keystone/keystone/notifications.py:230}} Jul 13 22:20:37.174882 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-e193fd4a-3e90-46e1-a4a4-329b28727682 admin admin] Invoking callback _drop_receipt_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:20:37.175602 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.notifications [None req-e193fd4a-3e90-46e1-a4a4-329b28727682 admin admin] Invoking callback _drop_token_cache for event identity invalidate_token_cache internal for {'resource_info': None} {{(pid=80381) notify_event_callbacks /opt/stack/keystone/keystone/notifications.py:355}} Jul 13 22:20:37.177480 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1219/2431] 10.222.0.52 () {58 vars in 1232 bytes} [Wed Jul 13 22:20:37 2022] DELETE /identity/v3/projects/614cb1cb24bd4bc8b7c48e5f47b501ca => generated 0 bytes in 121 msecs (HTTP/1.1 204) 3 headers in 132 bytes (0 switches on core 0) Jul 13 22:20:39.753125 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f28b04f-6577-4981-b981-9d3b01b3aef9 None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:20:39.753595 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f28b04f-6577-4981-b981-9d3b01b3aef9 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:20:39.753711 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5f28b04f-6577-4981-b981-9d3b01b3aef9 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:20:39.771134 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-5f28b04f-6577-4981-b981-9d3b01b3aef9 None None] MFA Rules not processed for user `064dcac8f59c431cb7d6848f4aaac347`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 22:20:39.772335 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-5f28b04f-6577-4981-b981-9d3b01b3aef9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:20:39.828371 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1213/2432] 10.222.0.52 () {58 vars in 957 bytes} [Wed Jul 13 22:20:39 2022] POST /identity/v3/auth/tokens => generated 3252 bytes in 77 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 13 22:20:40.143828 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e193fd4a-3e90-46e1-a4a4-329b28727682 admin admin] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:20:40.145864 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-e193fd4a-3e90-46e1-a4a4-329b28727682 admin admin] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:20:40.205025 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-e193fd4a-3e90-46e1-a4a4-329b28727682 admin admin] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:20:40.208825 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-08f76caa-4790-4bda-a3fc-5d0749c26695 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '847ec9a1bd4d4dec804318a5c845895a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417e3e2f1333481795a089bef2e24bea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:20:40.209479 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f76caa-4790-4bda-a3fc-5d0749c26695 service neutron] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:20:40.209762 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f76caa-4790-4bda-a3fc-5d0749c26695 service neutron] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:20:40.210061 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-08f76caa-4790-4bda-a3fc-5d0749c26695 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:20:40.211912 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-08f76caa-4790-4bda-a3fc-5d0749c26695 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:20:40.222988 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f76caa-4790-4bda-a3fc-5d0749c26695 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:20:40.224454 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-08f76caa-4790-4bda-a3fc-5d0749c26695 service neutron] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:20:40.233347 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1220/2433] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jul 13 22:20:40 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 90 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 22:20:40.806810 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f253c16-539f-4972-b306-4918b5fe83f9 None None] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:20:40.807381 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f253c16-539f-4972-b306-4918b5fe83f9 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:20:40.807381 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-3f253c16-539f-4972-b306-4918b5fe83f9 None None] PATH_INFO: `/` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:20:40.808794 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1214/2434] 10.222.0.52 () {54 vars in 921 bytes} [Wed Jul 13 22:20:40 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 22:20:40.814866 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3688a27-5d6b-4e7a-88ad-1338bfb8d80d None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:20:40.815157 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3688a27-5d6b-4e7a-88ad-1338bfb8d80d None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:20:40.815420 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f3688a27-5d6b-4e7a-88ad-1338bfb8d80d None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:20:40.849041 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-f3688a27-5d6b-4e7a-88ad-1338bfb8d80d None None] MFA Rules not processed for user `f84281091687464990ffbca590b9343b`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 22:20:40.850268 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-f3688a27-5d6b-4e7a-88ad-1338bfb8d80d None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:20:40.932912 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1221/2435] 10.222.0.52 () {58 vars in 1018 bytes} [Wed Jul 13 22:20:40 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 119 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 22:20:40.945951 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f253c16-539f-4972-b306-4918b5fe83f9 None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:20:40.947843 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-3f253c16-539f-4972-b306-4918b5fe83f9 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:20:41.007847 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-3f253c16-539f-4972-b306-4918b5fe83f9 None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:20:41.011638 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-f04877ef-e502-4013-b345-e645e348f6af service glance] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'e4d35d50e15d43d3afc59206b8165f30', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417e3e2f1333481795a089bef2e24bea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:20:41.012310 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04877ef-e502-4013-b345-e645e348f6af service glance] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:20:41.012644 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04877ef-e502-4013-b345-e645e348f6af service glance] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:20:41.012889 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-f04877ef-e502-4013-b345-e645e348f6af service glance] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:20:41.024043 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04877ef-e502-4013-b345-e645e348f6af service glance] RBAC: Authorizing `identity:validate_token()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:20:41.025420 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-f04877ef-e502-4013-b345-e645e348f6af service glance] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:20:41.036396 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1215/2436] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jul 13 22:20:40 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 91 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 22:20:41.107572 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b3b18f-8837-42fd-8ebb-41dc5df0e6c6 None None] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:20:41.107879 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b3b18f-8837-42fd-8ebb-41dc5df0e6c6 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:20:41.108153 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-c2b3b18f-8837-42fd-8ebb-41dc5df0e6c6 None None] PATH_INFO: `/` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:20:41.109544 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1222/2437] 10.222.0.52 () {54 vars in 921 bytes} [Wed Jul 13 22:20:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 22:20:41.115430 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2440433a-6665-4fe8-8751-d6a2b1ba1dec None None] REQUEST_METHOD: `POST` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:20:41.115739 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2440433a-6665-4fe8-8751-d6a2b1ba1dec None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:20:41.116105 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-2440433a-6665-4fe8-8751-d6a2b1ba1dec None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:20:41.138630 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.auth.core [None req-2440433a-6665-4fe8-8751-d6a2b1ba1dec None None] MFA Rules not processed for user `847ec9a1bd4d4dec804318a5c845895a`. Rule list: `[]` (Enabled: `True`). {{(pid=80382) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 22:20:41.139846 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-2440433a-6665-4fe8-8751-d6a2b1ba1dec None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:20:41.147434 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1216/2438] 10.222.0.52 () {58 vars in 1018 bytes} [Wed Jul 13 22:20:41 2022] POST /identity/v3/auth/tokens => generated 3111 bytes in 34 msecs (HTTP/1.1 201) 6 headers in 385 bytes (2 switches on core 0) Jul 13 22:20:41.160079 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b3b18f-8837-42fd-8ebb-41dc5df0e6c6 None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:20:41.162702 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-c2b3b18f-8837-42fd-8ebb-41dc5df0e6c6 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:20:41.178300 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-c2b3b18f-8837-42fd-8ebb-41dc5df0e6c6 None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:20:41.182655 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-7a685992-6008-495a-9bf4-bd50cde3ed76 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '847ec9a1bd4d4dec804318a5c845895a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417e3e2f1333481795a089bef2e24bea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:20:41.183301 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a685992-6008-495a-9bf4-bd50cde3ed76 service neutron] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:20:41.183584 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a685992-6008-495a-9bf4-bd50cde3ed76 service neutron] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:20:41.183854 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-7a685992-6008-495a-9bf4-bd50cde3ed76 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:20:41.191756 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a685992-6008-495a-9bf4-bd50cde3ed76 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:20:41.193206 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-7a685992-6008-495a-9bf4-bd50cde3ed76 service neutron] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:20:41.201755 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1223/2439] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jul 13 22:20:41 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 43 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 13 22:20:41.289989 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f5f1ff-f7db-4adf-a886-1844dad508a5 None None] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:20:41.290274 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f5f1ff-f7db-4adf-a886-1844dad508a5 None None] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:20:41.290528 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-a1f5f1ff-f7db-4adf-a886-1844dad508a5 None None] PATH_INFO: `/` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:20:41.291976 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1217/2440] 10.222.0.52 () {54 vars in 921 bytes} [Wed Jul 13 22:20:41 2022] GET /identity => generated 270 bytes in 3 msecs (HTTP/1.1 300) 6 headers in 234 bytes (1 switches on core 0) Jul 13 22:20:41.297533 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8f3211-7748-4f1e-be42-35bc9ca1d62f None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:20:41.297811 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8f3211-7748-4f1e-be42-35bc9ca1d62f None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:20:41.298150 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5b8f3211-7748-4f1e-be42-35bc9ca1d62f None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:20:41.318227 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-5b8f3211-7748-4f1e-be42-35bc9ca1d62f None None] MFA Rules not processed for user `f84281091687464990ffbca590b9343b`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 22:20:41.319523 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-5b8f3211-7748-4f1e-be42-35bc9ca1d62f None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:20:41.331751 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1224/2441] 10.222.0.52 () {58 vars in 1018 bytes} [Wed Jul 13 22:20:41 2022] POST /identity/v3/auth/tokens => generated 3362 bytes in 35 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 22:20:41.343635 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f5f1ff-f7db-4adf-a886-1844dad508a5 None None] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:20:41.351433 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:DELETE=4,keystone:SELECT=95,keystone:INSERT=1 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:20:41.356605 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-a1f5f1ff-f7db-4adf-a886-1844dad508a5 None None] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:20:41.361550 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8dd2d3-f461-467b-a1fc-1105c893d2f5 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '847ec9a1bd4d4dec804318a5c845895a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417e3e2f1333481795a089bef2e24bea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:20:41.362400 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8dd2d3-f461-467b-a1fc-1105c893d2f5 service neutron] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:20:41.362763 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8dd2d3-f461-467b-a1fc-1105c893d2f5 service neutron] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:20:41.363245 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-fd8dd2d3-f461-467b-a1fc-1105c893d2f5 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:20:41.375040 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8dd2d3-f461-467b-a1fc-1105c893d2f5 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:20:41.376596 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-fd8dd2d3-f461-467b-a1fc-1105c893d2f5 service neutron] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:20:41.388115 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1218/2442] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jul 13 22:20:41 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 45 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 22:20:43.873806 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f67d92c-f07a-4abf-a08b-3cc9d447d90c None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:20:43.874276 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f67d92c-f07a-4abf-a08b-3cc9d447d90c None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:20:43.874367 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-8f67d92c-f07a-4abf-a08b-3cc9d447d90c None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:20:43.894883 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-8f67d92c-f07a-4abf-a08b-3cc9d447d90c None None] MFA Rules not processed for user `b866bacdafae4db6a32714c07c56515f`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 22:20:43.896453 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-8f67d92c-f07a-4abf-a08b-3cc9d447d90c None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:20:43.926479 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=95,keystone:DELETE=4,keystone:INSERT=1 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:20:43.970149 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1225/2443] 10.222.0.52 () {58 vars in 1009 bytes} [Wed Jul 13 22:20:43 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 97 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 22:20:43.983637 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8dd2d3-f461-467b-a1fc-1105c893d2f5 service neutron] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:20:43.985871 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-fd8dd2d3-f461-467b-a1fc-1105c893d2f5 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:20:44.070890 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-fd8dd2d3-f461-467b-a1fc-1105c893d2f5 service neutron] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:20:44.075820 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a28f40-31fa-4822-9540-26890b1a4208 service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eb1525233ba48c99b15d4cb75db4822', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417e3e2f1333481795a089bef2e24bea', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:20:44.076480 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a28f40-31fa-4822-9540-26890b1a4208 service swift] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:20:44.076758 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a28f40-31fa-4822-9540-26890b1a4208 service swift] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:20:44.077034 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-32a28f40-31fa-4822-9540-26890b1a4208 service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:20:44.085617 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a28f40-31fa-4822-9540-26890b1a4208 service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:20:44.087206 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-32a28f40-31fa-4822-9540-26890b1a4208 service swift] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:20:44.096501 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1219/2444] 10.222.0.52 () {60 vars in 1384 bytes} [Wed Jul 13 22:20:43 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 113 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 13 22:20:53.962963 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=14 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:20:54.093048 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=36 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:21:05.955929 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c02512-81af-47a0-9cd0-50c24815eba5 None None] REQUEST_METHOD: `POST` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:21:05.956475 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c02512-81af-47a0-9cd0-50c24815eba5 None None] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:21:05.956475 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-d1c02512-81af-47a0-9cd0-50c24815eba5 None None] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:21:05.975382 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.auth.core [None req-d1c02512-81af-47a0-9cd0-50c24815eba5 None None] MFA Rules not processed for user `b866bacdafae4db6a32714c07c56515f`. Rule list: `[]` (Enabled: `True`). {{(pid=80381) check_auth_methods_against_rules /opt/stack/keystone/keystone/auth/core.py:438}} Jul 13 22:21:05.976592 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-d1c02512-81af-47a0-9cd0-50c24815eba5 None None] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:21:05.984226 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1226/2445] 10.222.0.52 () {58 vars in 967 bytes} [Wed Jul 13 22:21:05 2022] POST /identity/v3/auth/tokens => generated 3185 bytes in 29 msecs (HTTP/1.1 201) 6 headers in 385 bytes (1 switches on core 0) Jul 13 22:21:05.997206 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a28f40-31fa-4822-9540-26890b1a4208 service swift] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:21:06.008347 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-32a28f40-31fa-4822-9540-26890b1a4208 service swift] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:21:06.020937 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53aed940-472e-40e7-aa05-78ebe95febec service swift] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '1eb1525233ba48c99b15d4cb75db4822', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417e3e2f1333481795a089bef2e24bea', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:21:06.021844 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53aed940-472e-40e7-aa05-78ebe95febec service swift] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:21:06.022119 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53aed940-472e-40e7-aa05-78ebe95febec service swift] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:21:06.022450 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-53aed940-472e-40e7-aa05-78ebe95febec service swift] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:21:06.032670 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53aed940-472e-40e7-aa05-78ebe95febec service swift] RBAC: Authorizing `identity:validate_token()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:21:06.035009 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-53aed940-472e-40e7-aa05-78ebe95febec service swift] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:21:06.043582 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1220/2446] 10.222.0.52 () {60 vars in 1384 bytes} [Wed Jul 13 22:21:05 2022] GET /identity/v3/auth/tokens?nocatalog => generated 604 bytes in 47 msecs (HTTP/1.1 200) 6 headers in 379 bytes (1 switches on core 0) Jul 13 22:21:15.970119 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=3 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:21:16.040381 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=6 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:22:06.743797 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c02512-81af-47a0-9cd0-50c24815eba5 None None] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:22:06.754834 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-d1c02512-81af-47a0-9cd0-50c24815eba5 None None] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:22:06.758965 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc45c01-9b0c-4c2d-97f0-459c230e0a15 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '847ec9a1bd4d4dec804318a5c845895a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417e3e2f1333481795a089bef2e24bea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:22:06.759642 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc45c01-9b0c-4c2d-97f0-459c230e0a15 service neutron] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:22:06.759911 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc45c01-9b0c-4c2d-97f0-459c230e0a15 service neutron] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:22:06.760181 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-adc45c01-9b0c-4c2d-97f0-459c230e0a15 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:22:06.762179 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-adc45c01-9b0c-4c2d-97f0-459c230e0a15 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:22:06.776421 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc45c01-9b0c-4c2d-97f0-459c230e0a15 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:22:06.777916 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-adc45c01-9b0c-4c2d-97f0-459c230e0a15 service neutron] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:22:06.789700 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1227/2447] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jul 13 22:22:06 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 22:22:12.576110 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53aed940-472e-40e7-aa05-78ebe95febec service swift] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:22:12.585984 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-53aed940-472e-40e7-aa05-78ebe95febec service swift] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:22:12.589697 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed66045-15d3-4ba0-8ba7-4ffb1e6550bb service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '847ec9a1bd4d4dec804318a5c845895a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417e3e2f1333481795a089bef2e24bea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:22:12.590337 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed66045-15d3-4ba0-8ba7-4ffb1e6550bb service neutron] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:22:12.590656 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed66045-15d3-4ba0-8ba7-4ffb1e6550bb service neutron] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:22:12.590926 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-5ed66045-15d3-4ba0-8ba7-4ffb1e6550bb service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:22:12.592676 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-5ed66045-15d3-4ba0-8ba7-4ffb1e6550bb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:22:12.659566 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed66045-15d3-4ba0-8ba7-4ffb1e6550bb service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:22:12.661085 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-5ed66045-15d3-4ba0-8ba7-4ffb1e6550bb service neutron] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:22:12.671339 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1221/2448] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jul 13 22:22:12 2022] GET /identity/v3/auth/tokens => generated 3229 bytes in 96 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 13 22:22:13.703460 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc45c01-9b0c-4c2d-97f0-459c230e0a15 service neutron] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:22:13.713996 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-adc45c01-9b0c-4c2d-97f0-459c230e0a15 service neutron] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:22:13.718604 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1c8b45-2f97-4eb2-a9c4-2bbc132812a9 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '847ec9a1bd4d4dec804318a5c845895a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417e3e2f1333481795a089bef2e24bea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:22:13.719575 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1c8b45-2f97-4eb2-a9c4-2bbc132812a9 service neutron] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:22:13.719757 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1c8b45-2f97-4eb2-a9c4-2bbc132812a9 service neutron] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:22:13.719976 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-dd1c8b45-2f97-4eb2-a9c4-2bbc132812a9 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:22:13.721898 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-dd1c8b45-2f97-4eb2-a9c4-2bbc132812a9 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:22:13.736061 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1c8b45-2f97-4eb2-a9c4-2bbc132812a9 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:22:13.737506 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-dd1c8b45-2f97-4eb2-a9c4-2bbc132812a9 service neutron] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:22:13.748660 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1228/2449] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jul 13 22:22:13 2022] GET /identity/v3/auth/tokens => generated 3362 bytes in 46 msecs (HTTP/1.1 200) 6 headers in 380 bytes (2 switches on core 0) Jul 13 22:22:16.335465 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed66045-15d3-4ba0-8ba7-4ffb1e6550bb service neutron] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:22:16.337626 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-5ed66045-15d3-4ba0-8ba7-4ffb1e6550bb service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:22:16.456521 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-5ed66045-15d3-4ba0-8ba7-4ffb1e6550bb service neutron] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:22:16.462821 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3eeb2a-617c-45dc-9fa7-c1336ea994c2 service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac673d641d8849c79bf4cf71c9805587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417e3e2f1333481795a089bef2e24bea', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:22:16.463862 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3eeb2a-617c-45dc-9fa7-c1336ea994c2 service placement] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:22:16.464340 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3eeb2a-617c-45dc-9fa7-c1336ea994c2 service placement] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:22:16.464834 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-4a3eeb2a-617c-45dc-9fa7-c1336ea994c2 service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:22:16.467471 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-4a3eeb2a-617c-45dc-9fa7-c1336ea994c2 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:22:16.481788 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a3eeb2a-617c-45dc-9fa7-c1336ea994c2 service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:22:16.485390 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-4a3eeb2a-617c-45dc-9fa7-c1336ea994c2 service placement] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:22:16.497624 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1222/2450] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jul 13 22:22:16 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 163 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 22:22:23.744077 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=12 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:22:26.493187 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=62 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:23:20.206673 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1c8b45-2f97-4eb2-a9c4-2bbc132812a9 service neutron] Authenticating user token {{(pid=80381) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:23:20.216393 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-dd1c8b45-2f97-4eb2-a9c4-2bbc132812a9 service neutron] Validating token access rules against request {{(pid=80381) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:23:20.220912 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0bc96e10-6ffa-40c4-add6-9e74d47f6af2 service neutron] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': '847ec9a1bd4d4dec804318a5c845895a', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417e3e2f1333481795a089bef2e24bea', 'project_domain_id': 'default', 'roles': ['service'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80381) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:23:20.221556 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc96e10-6ffa-40c4-add6-9e74d47f6af2 service neutron] REQUEST_METHOD: `GET` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:23:20.221845 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc96e10-6ffa-40c4-add6-9e74d47f6af2 service neutron] SCRIPT_NAME: `/identity` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:23:20.222079 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0bc96e10-6ffa-40c4-add6-9e74d47f6af2 service neutron] PATH_INFO: `/v3/auth/tokens` {{(pid=80381) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:23:20.224472 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.fernet_utils [None req-0bc96e10-6ffa-40c4-add6-9e74d47f6af2 service neutron] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80381) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:23:20.237365 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc96e10-6ffa-40c4-add6-9e74d47f6af2 service neutron] RBAC: Authorizing `identity:validate_token()` {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:23:20.238858 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0bc96e10-6ffa-40c4-add6-9e74d47f6af2 service neutron] RBAC: Authorization granted {{(pid=80381) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:23:20.248204 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: [pid: 80381|app: 0|req: 1229/2451] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jul 13 22:23:20 2022] GET /identity/v3/auth/tokens => generated 3111 bytes in 42 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 22:23:30.244274 dv-cinder-799871-28-iscsi devstack@keystone.service[80381]: DEBUG dbcounter [-] [80381] Writing DB stats keystone:SELECT=6 {{(pid=80381) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}} Jul 13 22:25:24.723874 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3eeb2a-617c-45dc-9fa7-c1336ea994c2 service placement] Authenticating user token {{(pid=80382) process_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:407}} Jul 13 22:25:24.725504 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-4a3eeb2a-617c-45dc-9fa7-c1336ea994c2 service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:25:24.739302 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-4a3eeb2a-617c-45dc-9fa7-c1336ea994c2 service placement] Validating token access rules against request {{(pid=80382) validate_allowed_request /usr/local/lib/python3.8/dist-packages/keystonemiddleware/auth_token/__init__.py:545}} Jul 13 22:25:24.746321 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.middleware.auth_context [None req-0049f0ec-a6a0-4e7c-b3f8-76234c0395fe service placement] RBAC: auth_context: {'token': , 'domain_id': None, 'trust_id': None, 'trustor_id': None, 'trustee_id': None, 'domain_name': None, 'group_ids': [], 'user_id': 'ac673d641d8849c79bf4cf71c9805587', 'user_domain_id': 'default', 'system_scope': None, 'project_id': '417e3e2f1333481795a089bef2e24bea', 'project_domain_id': 'default', 'roles': ['service', 'reader', 'admin', 'member'], 'is_admin_project': True, 'service_user_id': None, 'service_user_domain_id': None, 'service_project_id': None, 'service_project_domain_id': None, 'service_roles': []} {{(pid=80382) fill_context /opt/stack/keystone/keystone/server/flask/request_processing/middleware/auth_context.py:478}} Jul 13 22:25:24.746996 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0049f0ec-a6a0-4e7c-b3f8-76234c0395fe service placement] REQUEST_METHOD: `GET` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:27}} Jul 13 22:25:24.747282 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0049f0ec-a6a0-4e7c-b3f8-76234c0395fe service placement] SCRIPT_NAME: `/identity` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:28}} Jul 13 22:25:24.747555 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.server.flask.request_processing.req_logging [None req-0049f0ec-a6a0-4e7c-b3f8-76234c0395fe service placement] PATH_INFO: `/v3/auth/tokens` {{(pid=80382) log_request_info /opt/stack/keystone/keystone/server/flask/request_processing/req_logging.py:29}} Jul 13 22:25:24.749978 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.fernet_utils [None req-0049f0ec-a6a0-4e7c-b3f8-76234c0395fe service placement] Loaded 2 Fernet keys from /etc/keystone/fernet-keys/, but `[fernet_tokens] max_active_keys = 3`; perhaps there have not been enough key rotations to reach `max_active_keys` yet? {{(pid=80382) load_keys /opt/stack/keystone/keystone/common/fernet_utils.py:286}} Jul 13 22:25:24.762899 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0049f0ec-a6a0-4e7c-b3f8-76234c0395fe service placement] RBAC: Authorizing `identity:validate_token()` {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:449}} Jul 13 22:25:24.764500 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG keystone.common.rbac_enforcer.enforcer [None req-0049f0ec-a6a0-4e7c-b3f8-76234c0395fe service placement] RBAC: Authorization granted {{(pid=80382) enforce_call /opt/stack/keystone/keystone/common/rbac_enforcer/enforcer.py:457}} Jul 13 22:25:24.775043 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: [pid: 80382|app: 0|req: 1223/2452] 10.222.0.52 () {60 vars in 1365 bytes} [Wed Jul 13 22:25:24 2022] GET /identity/v3/auth/tokens => generated 3298 bytes in 52 msecs (HTTP/1.1 200) 6 headers in 380 bytes (1 switches on core 0) Jul 13 22:25:34.770878 dv-cinder-799871-28-iscsi devstack@keystone.service[80382]: DEBUG dbcounter [-] [80382] Writing DB stats keystone:SELECT=6 {{(pid=80382) stat_writer /usr/local/lib/python3.8/dist-packages/dbcounter.py:114}}